{"id":"140d4839-13a6-4832-95c3-22379efbdfe0","shortId":"cTByNJ","kind":"skill","title":"robot-perception","tagline":"Comprehensive best practices for robot perception systems covering cameras, LiDARs, depth sensors, IMUs, and multi-sensor setups. Use this skill when working with RGB image processing, depth maps, point clouds, sensor calibration (intrinsic, extrinsic, hand-eye), object detection","description":"# Robot Perception Skill\n\n## When to Use This Skill\n- Setting up and configuring camera, LiDAR, or depth sensors\n- Building RGB, depth, or point cloud processing pipelines\n- Calibrating cameras (intrinsic, extrinsic, hand-eye)\n- Implementing object detection, segmentation, or tracking for robots\n- Fusing data from multiple sensor modalities\n- Streaming sensor data with proper threading and buffering\n- Synchronizing multi-sensor rigs\n- Deploying perception models on robot hardware (GPU, edge)\n- Debugging perception failures (latency, dropped frames, misalignment)\n\n## Sensor Landscape\n\n### Sensor Types and Characteristics\n\n```\nSensor Type        Output              Range       Rate     Best For\n─────────────────────────────────────────────────────────────────────────\nRGB Camera         (H,W,3) uint8       ∞           30-120Hz Object detection, tracking, visual servoing\nStereo Camera      (H,W,3)+(H,W,3)    0.3-20m     30-90Hz  Dense depth from passive stereo\nStructured Light   (H,W) float + RGB   0.2-10m     30Hz     Indoor manipulation, short range\nToF Depth          (H,W) float + RGB   0.1-10m     30Hz     Indoor, medium range\nLiDAR (spinning)   (N,3) or (N,4)     0.5-200m    10-20Hz  Outdoor navigation, mapping\nLiDAR (solid-st.)  (N,3)              0.5-200m    10-30Hz  Automotive, outdoor\nIMU                (6,) or (9,)        N/A         200-1kHz Orientation, motion estimation\nForce/Torque       (6,) float          N/A         1kHz+    Contact detection, force control\nTactile            (H,W) or (N,3)      Contact     30-100Hz Grasp quality, texture\nEvent Camera       Events (x,y,t,p)    ∞           μs       High-speed tracking, HDR scenes\n```\n\n### Common Sensor Hardware\n\n```\nDevice             Type               SDK/Driver           ROS2 Package\n──────────────────────────────────────────────────────────────────────────\nIntel RealSense    Structured Light   pyrealsense2         realsense2_camera\nStereolabs ZED     Stereo + IMU       pyzed                zed_wrapper\nLuxonis OAK-D      Stereo + Neural    depthai              depthai_ros\nFLIR/Basler        Industrial RGB     PySpin/pypylon       spinnaker_camera_driver\nVelodyne           Spinning LiDAR     velodyne_driver      velodyne\nOuster             Spinning LiDAR     ouster-sdk           ros2_ouster\nLivox              Solid-state LiDAR  livox_sdk            livox_ros2_driver\nUSB Webcam         RGB                OpenCV VideoCapture  usb_cam / v4l2_camera\n```\n\n## Camera Models and Calibration\n\n### Pinhole Camera Model\n\n```\n                    3D World Point (X, Y, Z)\n                           |\n                    [R | t] — Extrinsic (world → camera)\n                           |\n                    Camera Point (Xc, Yc, Zc)\n                           |\n                    K — Intrinsic (camera → pixel)\n                           |\n                    Pixel (u, v)\n\nK = [ fx   0   cx ]      fx, fy = focal lengths (pixels)\n    [  0  fy   cy ]      cx, cy = principal point\n    [  0   0    1 ]\n\nProjection:  [u, v, 1]^T = K @ [R | t] @ [X, Y, Z, 1]^T\n```\n\n### Intrinsic Calibration\n\n```python\nimport cv2\nimport numpy as np\nfrom pathlib import Path\n\nclass IntrinsicCalibrator:\n    \"\"\"Camera intrinsic calibration using checkerboard pattern\"\"\"\n\n    def __init__(self, board_size=(9, 6), square_size_m=0.025):\n        self.board_size = board_size\n        self.square_size = square_size_m\n\n        # Prepare object points (3D coordinates of checkerboard corners)\n        self.objp = np.zeros((board_size[0] * board_size[1], 3), np.float32)\n        self.objp[:, :2] = np.mgrid[\n            0:board_size[0], 0:board_size[1]\n        ].T.reshape(-1, 2) * square_size_m\n\n    def collect_calibration_images(self, camera, num_images=30,\n                                    min_coverage=0.6):\n        \"\"\"Collect calibration images with good spatial coverage.\n\n        IMPORTANT: Move the board to cover all regions of the image,\n        including corners and edges. Tilt the board at various angles.\n        Bad coverage = bad calibration, especially at image edges.\n        \"\"\"\n        obj_points = []\n        img_points = []\n        coverage_map = np.zeros((4, 4), dtype=int)  # Track board positions\n\n        while len(obj_points) < num_images:\n            frame = camera.capture()\n            gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n\n            found, corners = cv2.findChessboardCorners(\n                gray, self.board_size,\n                cv2.CALIB_CB_ADAPTIVE_THRESH |\n                cv2.CALIB_CB_NORMALIZE_IMAGE |\n                cv2.CALIB_CB_FAST_CHECK\n            )\n\n            if found:\n                # Sub-pixel refinement — critical for accuracy\n                criteria = (cv2.TERM_CRITERIA_EPS +\n                           cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)\n                corners = cv2.cornerSubPix(\n                    gray, corners, (11, 11), (-1, -1), criteria)\n\n                # Track coverage\n                center = corners.mean(axis=0).flatten()\n                grid_x = int(center[0] / gray.shape[1] * 4)\n                grid_y = int(center[1] / gray.shape[0] * 4)\n                grid_x = min(grid_x, 3)\n                grid_y = min(grid_y, 3)\n                coverage_map[grid_y, grid_x] += 1\n\n                obj_points.append(self.objp)\n                img_points.append(corners)\n\n        coverage = (coverage_map > 0).sum() / coverage_map.size\n        if coverage < min_coverage:\n            print(f\"WARNING: Only {coverage:.0%} coverage. \"\n                  f\"Move board to uncovered regions.\")\n\n        return obj_points, img_points, gray.shape[::-1]\n\n    def calibrate(self, obj_points, img_points, image_size):\n        \"\"\"Run calibration and return camera matrix + distortion coeffs\"\"\"\n        ret, K, dist, rvecs, tvecs = cv2.calibrateCamera(\n            obj_points, img_points, image_size, None, None)\n\n        if ret > 1.0:\n            print(f\"WARNING: High reprojection error ({ret:.3f} px). \"\n                  f\"Check image quality and board detection.\")\n\n        # Compute per-image reprojection errors\n        errors = []\n        for i in range(len(obj_points)):\n            projected, _ = cv2.projectPoints(\n                obj_points[i], rvecs[i], tvecs[i], K, dist)\n            err = cv2.norm(img_points[i], projected, cv2.NORM_L2)\n            err /= len(projected)\n            errors.append(err)\n\n        print(f\"Calibration complete:\")\n        print(f\"  RMS reprojection error: {ret:.4f} px\")\n        print(f\"  Per-image errors: mean={np.mean(errors):.4f}, \"\n              f\"max={np.max(errors):.4f}\")\n        print(f\"  Focal length: fx={K[0,0]:.1f}, fy={K[1,1]:.1f}\")\n        print(f\"  Principal point: cx={K[0,2]:.1f}, cy={K[1,2]:.1f}\")\n\n        return CalibrationResult(\n            camera_matrix=K, dist_coeffs=dist,\n            rms_error=ret, image_size=image_size)\n\n    def save(self, result, path):\n        \"\"\"Save calibration to YAML (OpenCV-compatible format)\"\"\"\n        fs = cv2.FileStorage(str(path), cv2.FILE_STORAGE_WRITE)\n        fs.write(\"camera_matrix\", result.camera_matrix)\n        fs.write(\"dist_coeffs\", result.dist_coeffs)\n        fs.write(\"image_width\", result.image_size[0])\n        fs.write(\"image_height\", result.image_size[1])\n        fs.write(\"rms_error\", result.rms_error)\n        fs.release()\n\n    @staticmethod\n    def load(path):\n        \"\"\"Load calibration from YAML\"\"\"\n        fs = cv2.FileStorage(str(path), cv2.FILE_STORAGE_READ)\n        K = fs.getNode(\"camera_matrix\").mat()\n        dist = fs.getNode(\"dist_coeffs\").mat()\n        w = int(fs.getNode(\"image_width\").real())\n        h = int(fs.getNode(\"image_height\").real())\n        fs.release()\n        return CalibrationResult(\n            camera_matrix=K, dist_coeffs=dist,\n            image_size=(w, h), rms_error=0.0)\n```\n\n### Extrinsic Calibration (Camera-to-Camera, Camera-to-LiDAR)\n\n```python\nclass ExtrinsicCalibrator:\n    \"\"\"Compute transform between two sensors using shared targets\"\"\"\n\n    def calibrate_stereo(self, calib_left, calib_right,\n                          obj_points, img_points_left, img_points_right,\n                          image_size):\n        \"\"\"Stereo calibration: find relative pose between two cameras\"\"\"\n        ret, K1, d1, K2, d2, R, T, E, F = cv2.stereoCalibrate(\n            obj_points, img_points_left, img_points_right,\n            calib_left.camera_matrix, calib_left.dist_coeffs,\n            calib_right.camera_matrix, calib_right.dist_coeffs,\n            image_size,\n            flags=cv2.CALIB_FIX_INTRINSIC  # Use pre-calibrated intrinsics\n        )\n\n        print(f\"Stereo calibration RMS: {ret:.4f} px\")\n        print(f\"Baseline: {np.linalg.norm(T):.4f} m\")\n\n        return StereoCalibration(R=R, T=T, E=E, F=F, rms_error=ret)\n\n    def calibrate_camera_to_lidar(self, camera_points_2d,\n                                    lidar_points_3d, K, dist):\n        \"\"\"Find camera-to-LiDAR transform using corresponding points.\n\n        Use a calibration target visible to both sensors (e.g.,\n        checkerboard with reflective tape corners).\n        \"\"\"\n        # PnP: find pose of 3D points relative to camera\n        success, rvec, tvec = cv2.solvePnP(\n            lidar_points_3d, camera_points_2d, K, dist,\n            flags=cv2.SOLVEPNP_ITERATIVE\n        )\n\n        if not success:\n            raise CalibrationError(\"PnP failed — check point correspondences\")\n\n        R, _ = cv2.Rodrigues(rvec)\n        T_camera_lidar = np.eye(4)\n        T_camera_lidar[:3, :3] = R\n        T_camera_lidar[:3, 3] = tvec.flatten()\n\n        # Verify by reprojecting\n        projected, _ = cv2.projectPoints(\n            lidar_points_3d, rvec, tvec, K, dist)\n        error = np.mean(np.linalg.norm(\n            camera_points_2d - projected.reshape(-1, 2), axis=1))\n        print(f\"Camera-LiDAR reprojection error: {error:.2f} px\")\n\n        return T_camera_lidar\n```\n\n### Hand-Eye Calibration (Camera-to-Robot)\n\n```python\nclass HandEyeCalibrator:\n    \"\"\"Solve AX = XB for camera mounted on robot end-effector (eye-in-hand)\n    or camera mounted on a fixed base (eye-to-hand).\n\n    Requires moving the robot to multiple poses while observing a\n    fixed calibration target.\n    \"\"\"\n\n    def __init__(self, K, dist, board_size=(9, 6), square_size=0.025):\n        self.K = K\n        self.dist = dist\n        self.board_size = board_size\n        self.square_size = square_size\n        self.objp = np.zeros((board_size[0] * board_size[1], 3), np.float32)\n        self.objp[:, :2] = np.mgrid[\n            0:board_size[0], 0:board_size[1]\n        ].T.reshape(-1, 2) * square_size\n\n    def collect_poses(self, camera, robot, num_poses=20):\n        \"\"\"Collect camera-target and robot poses at multiple configurations.\n\n        IMPORTANT: Move to diverse robot orientations. At least 3 different\n        rotation axes. Pure translations are NOT sufficient.\n        \"\"\"\n        R_gripper2base = []\n        t_gripper2base = []\n        R_target2cam = []\n        t_target2cam = []\n\n        for i in range(num_poses):\n            input(f\"Move robot to pose {i+1}/{num_poses}, press Enter...\")\n\n            # Get robot end-effector pose\n            ee_pose = robot.get_ee_pose()  # 4x4 homogeneous matrix\n            R_gripper2base.append(ee_pose[:3, :3])\n            t_gripper2base.append(ee_pose[:3, 3])\n\n            # Detect calibration target in camera\n            frame = camera.capture()\n            gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n            found, corners = cv2.findChessboardCorners(\n                gray, self.board_size, None)\n\n            if not found:\n                print(f\"  Board not detected at pose {i+1}, skip.\")\n                continue\n\n            corners = cv2.cornerSubPix(\n                gray, corners, (11, 11), (-1, -1),\n                (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001))\n\n            ret, rvec, tvec = cv2.solvePnP(\n                self.objp, corners, self.K, self.dist)\n            R, _ = cv2.Rodrigues(rvec)\n            R_target2cam.append(R)\n            t_target2cam.append(tvec.flatten())\n\n        return (R_gripper2base, t_gripper2base,\n                R_target2cam, t_target2cam)\n\n    def calibrate_eye_in_hand(self, R_g2b, t_g2b, R_t2c, t_t2c):\n        \"\"\"Eye-in-hand: camera mounted on end-effector.\n        Solves for T_camera_to_gripper.\"\"\"\n        R, t = cv2.calibrateHandEye(\n            R_g2b, t_g2b, R_t2c, t_t2c,\n            method=cv2.CALIB_HAND_EYE_TSAI  # Also: PARK, HORAUD, DANIILIDIS\n        )\n        T_cam2gripper = np.eye(4)\n        T_cam2gripper[:3, :3] = R\n        T_cam2gripper[:3, 3] = t.flatten()\n        return T_cam2gripper\n\n    def calibrate_eye_to_hand(self, R_g2b, t_g2b, R_t2c, t_t2c):\n        \"\"\"Eye-to-hand: camera fixed in workspace.\n        Solves for T_camera_to_base.\"\"\"\n        # Invert robot poses (base-to-gripper → gripper-to-base)\n        R_b2g = [R.T for R in R_g2b]\n        t_b2g = [-R.T @ t for R, t in zip(R_g2b, t_g2b)]\n\n        R, t = cv2.calibrateHandEye(\n            R_b2g, t_b2g, R_t2c, t_t2c,\n            method=cv2.CALIB_HAND_EYE_TSAI\n        )\n        T_cam2base = np.eye(4)\n        T_cam2base[:3, :3] = R\n        T_cam2base[:3, 3] = t.flatten()\n        return T_cam2base\n\n    def verify_calibration(self, T_cam2ee, robot, camera, target_points_3d):\n        \"\"\"Verify by projecting a known 3D point through the full chain.\n        Error should be < 5mm for manipulation tasks.\"\"\"\n        ee_pose = robot.get_ee_pose()\n        T_cam2base = ee_pose @ T_cam2ee\n\n        # Project world point to camera\n        point_in_cam = np.linalg.inv(T_cam2base) @ np.append(\n            target_points_3d[0], 1.0)\n        projected, _ = cv2.projectPoints(\n            point_in_cam[:3].reshape(1, 3),\n            np.zeros(3), np.zeros(3), self.K, self.dist)\n\n        print(f\"Verification projection: {projected.flatten()}\")\n        return projected.flatten()\n```\n\n### Calibration Quality Checklist\n\n```\n✅ Intrinsic calibration:\n   - RMS reprojection error < 0.5 px (good), < 0.3 px (excellent)\n   - ≥ 20 images with board covering full image area (corners + edges)\n   - Board tilted at multiple angles (not just fronto-parallel)\n   - Fixed focus / fixed zoom during calibration AND operation\n\n✅ Extrinsic calibration (stereo / camera-LiDAR):\n   - RMS < 1.0 px for stereo\n   - Reprojection error < 3 px for camera-LiDAR\n   - Verified with independent measurement (ruler / known distance)\n\n✅ Hand-eye calibration:\n   - ≥ 15 poses with diverse orientations (≥ 3 rotation axes)\n   - Verification error < 5mm for manipulation, < 10mm for navigation\n   - Robot repeatability accounted for (UR5 ≈ ±0.03mm, low-cost ≈ ±1mm)\n\n⚠️  Recalibrate when:\n   - Camera is physically bumped or remounted\n   - Lens focus or zoom is changed\n   - Temperature changes significantly (thermal expansion shifts intrinsics)\n   - Robot is re-homed or joints are recalibrated\n```\n\n## Sensor Streaming Best Practices\n\n### Camera Streaming Architecture\n\n```python\nimport threading\nimport time\nfrom collections import deque\nfrom dataclasses import dataclass, field\nfrom typing import Optional\nimport numpy as np\n\n@dataclass\nclass StampedFrame:\n    \"\"\"Frame with capture timestamp for synchronization\"\"\"\n    data: np.ndarray\n    timestamp: float          # time.monotonic() at capture\n    sequence: int             # Frame counter\n    sensor_id: str\n\nclass CameraStream:\n    \"\"\"Thread-safe camera streaming with bounded buffer.\n\n    Design principles:\n    1. Capture runs in a dedicated thread at sensor rate\n    2. Processing never blocks capture\n    3. Always use the LATEST frame (drop old ones for real-time)\n    4. Timestamp at capture, not at processing time\n    \"\"\"\n\n    def __init__(self, camera, buffer_size=2, name=\"camera\"):\n        self.camera = camera\n        self.name = name\n        self._buffer = deque(maxlen=buffer_size)\n        self._lock = threading.Lock()\n        self._new_frame = threading.Event()\n        self._running = False\n        self._sequence = 0\n        self._thread = None\n\n        # Diagnostics\n        self._capture_times = deque(maxlen=100)\n        self._drop_count = 0\n\n    def start(self):\n        \"\"\"Start capture thread\"\"\"\n        self._running = True\n        self._thread = threading.Thread(\n            target=self._capture_loop, daemon=True, name=f\"{self.name}_capture\")\n        self._thread.start()\n\n    def stop(self):\n        \"\"\"Stop capture thread\"\"\"\n        self._running = False\n        if self._thread:\n            self._thread.join(timeout=2.0)\n\n    def _capture_loop(self):\n        while self._running:\n            t_start = time.monotonic()\n\n            try:\n                raw = self.camera.capture()\n                timestamp = time.monotonic()  # Timestamp AFTER capture\n\n                frame = StampedFrame(\n                    data=raw,\n                    timestamp=timestamp,\n                    sequence=self._sequence,\n                    sensor_id=self.name\n                )\n                self._sequence += 1\n\n                with self._lock:\n                    if len(self._buffer) == self._buffer.maxlen:\n                        self._drop_count += 1\n                    self._buffer.append(frame)\n\n                self._new_frame.set()\n                self._capture_times.append(time.monotonic() - t_start)\n\n            except Exception as e:\n                print(f\"[{self.name}] Capture error: {e}\")\n                time.sleep(0.01)  # Back off on error\n\n    def get_latest(self) -> Optional[StampedFrame]:\n        \"\"\"Get most recent frame (non-blocking). Returns None if empty.\"\"\"\n        with self._lock:\n            if self._buffer:\n                return self._buffer[-1]\n        return None\n\n    def wait_for_frame(self, timeout=1.0) -> Optional[StampedFrame]:\n        \"\"\"Block until a new frame arrives\"\"\"\n        self._new_frame.clear()\n        if self._new_frame.wait(timeout=timeout):\n            return self.get_latest()\n        return None\n\n    def get_diagnostics(self) -> dict:\n        \"\"\"Streaming health metrics\"\"\"\n        if self._capture_times:\n            times = list(self._capture_times)\n            fps = 1.0 / np.mean(times) if np.mean(times) > 0 else 0\n        else:\n            fps = 0\n        return {\n            \"sensor\": self.name,\n            \"fps\": round(fps, 1),\n            \"frames_captured\": self._sequence,\n            \"frames_dropped\": self._drop_count,\n            \"buffer_size\": len(self._buffer),\n            \"avg_capture_ms\": round(np.mean(times) * 1000, 1) if self._capture_times else 0,\n        }\n```\n\n### Multi-Sensor Synchronization\n\n```python\nclass SyncedMultiSensor:\n    \"\"\"Synchronize frames from multiple sensors by timestamp.\n\n    Uses nearest-neighbor matching within a time tolerance.\n    For hardware-synced sensors, use hardware trigger instead.\n    \"\"\"\n\n    def __init__(self, sensors: dict, max_time_diff_ms=33):\n        \"\"\"\n        Args:\n            sensors: {\"rgb\": CameraStream, \"depth\": CameraStream, ...}\n            max_time_diff_ms: Maximum allowed time difference between\n                              synced frames. Default 33ms (1 frame at 30Hz).\n        \"\"\"\n        self.sensors = sensors\n        self.max_dt = max_time_diff_ms / 1000.0\n        self._synced_callback = None\n\n    def start(self):\n        for s in self.sensors.values():\n            s.start()\n\n    def stop(self):\n        for s in self.sensors.values():\n            s.stop()\n\n    def get_synced(self) -> Optional[dict]:\n        \"\"\"Get time-synchronized frames from all sensors.\n        Returns None if any sensor is missing or too far out of sync.\"\"\"\n        frames = {}\n        for name, stream in self.sensors.items():\n            frame = stream.get_latest()\n            if frame is None:\n                return None\n            frames[name] = frame\n\n        # Check time alignment against the first sensor\n        ref_time = list(frames.values())[0].timestamp\n        for name, frame in frames.items():\n            dt = abs(frame.timestamp - ref_time)\n            if dt > self.max_dt:\n                return None  # Out of sync\n\n        return frames\n\n    def get_synced_interpolated(self) -> Optional[dict]:\n        \"\"\"For sensors at different rates, interpolate to common timestamp.\n        Useful for IMU + camera fusion.\"\"\"\n        # Get latest from each sensor\n        frames = {}\n        for name, stream in self.sensors.items():\n            frame = stream.get_latest()\n            if frame is None:\n                return None\n            frames[name] = frame\n\n        # Use the SLOWEST sensor's timestamp as reference\n        ref_time = min(f.timestamp for f in frames.values())\n\n        result = {}\n        for name, frame in frames.items():\n            dt = frame.timestamp - ref_time\n            if abs(dt) <= self.max_dt:\n                result[name] = frame\n            # For high-rate sensors (IMU), could interpolate here\n\n        return result if len(result) == len(self.sensors) else None\n```\n\n### Hardware-Triggered Synchronization\n\n```python\nclass HardwareSyncConfig:\n    \"\"\"Configure hardware trigger for multi-camera synchronization.\n\n    ALWAYS prefer hardware sync over software sync for:\n    - Stereo depth computation\n    - Multi-camera 3D reconstruction\n    - Fast-moving scenes\n\n    Common trigger methods:\n    - GPIO pulse from microcontroller (Arduino, ESP32)\n    - Camera's own sync output → other cameras' trigger input\n    - PTP (Precision Time Protocol) for GigE cameras\n    \"\"\"\n\n    @staticmethod\n    def setup_realsense_sync(master_serial, slave_serials):\n        \"\"\"Configure RealSense hardware sync (Inter-Cam sync mode)\"\"\"\n        import pyrealsense2 as rs\n\n        # Master camera: generates sync signal\n        master = rs.pipeline()\n        master_cfg = rs.config()\n        master_cfg.enable_device(master_serial)\n        master_sensor = master.start(master_cfg)\n\n        # Set master to mode 1 (master)\n        depth_sensor = master_sensor.get_device().first_depth_sensor()\n        depth_sensor.set_option(rs.option.inter_cam_sync_mode, 1)\n\n        # Slave cameras: receive sync signal\n        slaves = []\n        for serial in slave_serials:\n            pipe = rs.pipeline()\n            cfg = rs.config()\n            cfg.enable_device(serial)\n            profile = pipe.start(cfg)\n            sensor = profile.get_device().first_depth_sensor()\n            sensor.set_option(rs.option.inter_cam_sync_mode, 2)  # Slave\n            slaves.append(pipe)\n\n        return master, slaves\n\n    @staticmethod\n    def setup_ptp_sync(camera_ips):\n        \"\"\"Enable PTP synchronization for GigE Vision cameras.\n\n        Requires PTP-capable network switch.\n        Achieves < 1μs sync accuracy.\n        \"\"\"\n        # Example with Basler/pylon cameras\n        # import pypylon.pylon as py\n        #\n        # for ip in camera_ips:\n        #     cam = py.InstantCamera(py.TlFactory.GetInstance()\n        #         .CreateDevice(py.CDeviceInfo().SetIpAddress(ip)))\n        #     cam.Open()\n        #     cam.GevIEEE1588.Value = True  # Enable PTP\n        #     cam.Close()\n        pass\n```\n\n### Streaming Anti-Patterns\n\n```python\n# ❌ BAD: Capture and process in same thread — limits to slowest operation\ndef bad_pipeline():\n    while True:\n        frame = camera.capture()           # 5ms\n        detections = model.detect(frame)   # 100ms\n        # Effective rate: ~9 FPS regardless of camera's 30 FPS\n\n# ✅ GOOD: Decouple capture from processing\ndef good_pipeline():\n    stream = CameraStream(camera)\n    stream.start()\n    while True:\n        frame = stream.get_latest()        # Always latest, non-blocking\n        if frame:\n            detections = model.detect(frame.data)  # 100ms\n            # Camera still capturing at 30 FPS in background\n            # Processing at ~10 FPS but always on freshest frame\n\n\n# ❌ BAD: Unbounded buffer — memory grows until OOM\nbuffer = []  # Will grow forever if consumer is slower than producer!\n\n# ✅ GOOD: Bounded buffer with drop policy\nbuffer = deque(maxlen=2)  # Keep latest 2, drop old automatically\n\n\n# ❌ BAD: Sleeping for frame timing\ntime.sleep(1.0 / 30)  # Drift accumulates, doesn't account for jitter\n\n# ✅ GOOD: Event-driven or spin with wall clock\nframe = stream.wait_for_frame(timeout=0.1)\n\n\n# ❌ BAD: Timestamp at processing time\ndef process_callback(raw_data):\n    timestamp = time.time()  # WRONG: includes queue delay + scheduling\n    frame = decode(raw_data)\n    publish(frame, timestamp)\n\n# ✅ GOOD: Timestamp at capture time\ndef capture_thread():\n    raw_data = sensor.read()\n    timestamp = time.monotonic()  # RIGHT: timestamp at source\n    queue.put((raw_data, timestamp))\n```\n\n## RGB Processing Pipelines\n\n### Image Undistortion\n\n```python\nclass ImageUndistorter:\n    \"\"\"Pre-compute undistortion maps for fast runtime correction.\n\n    ALWAYS undistort before any geometric computation\n    (pose estimation, 3D reconstruction, visual servoing).\n    \"\"\"\n\n    def __init__(self, K, dist, image_size, alpha=0.0):\n        \"\"\"\n        Args:\n            alpha: 0.0 = crop all black pixels (default, recommended)\n                   1.0 = keep all pixels (shows black borders)\n        \"\"\"\n        self.new_K, self.roi = cv2.getOptimalNewCameraMatrix(\n            K, dist, image_size, alpha, image_size)\n\n        # Pre-compute maps (do this ONCE, not per frame)\n        self.map1, self.map2 = cv2.initUndistortRectifyMap(\n            K, dist, None, self.new_K, image_size, cv2.CV_16SC2)\n\n    def undistort(self, image):\n        \"\"\"Apply undistortion using pre-computed maps (fast)\"\"\"\n        return cv2.remap(image, self.map1, self.map2,\n                         interpolation=cv2.INTER_LINEAR)\n\n    def undistort_points(self, points_2d):\n        \"\"\"Undistort 2D point coordinates\"\"\"\n        return cv2.undistortPoints(\n            points_2d.reshape(-1, 1, 2).astype(np.float64),\n            self.K, self.dist, P=self.new_K\n        ).reshape(-1, 2)\n```\n\n### Object Detection Integration\n\n```python\nclass RobotObjectDetector:\n    \"\"\"Object detection for robotic manipulation.\n\n    Wraps a detection model with robotics-specific post-processing:\n    - Workspace filtering (ignore detections outside robot reach)\n    - Stability tracking (reject flickering detections)\n    - 3D pose estimation from 2D detections + depth\n    \"\"\"\n\n    def __init__(self, model, K, workspace_bounds=None, min_confidence=0.5):\n        self.model = model\n        self.K = K\n        self.workspace_bounds = workspace_bounds\n        self.min_confidence = min_confidence\n        self.tracker = DetectionTracker(max_age=5, min_hits=3)\n\n    def detect(self, rgb, depth=None) -> list:\n        \"\"\"Run detection with robotics post-processing\"\"\"\n        # Raw detection\n        raw_dets = self.model(rgb)\n\n        # Filter by confidence\n        dets = [d for d in raw_dets if d.confidence >= self.min_confidence]\n\n        # Estimate 3D position if depth is available\n        if depth is not None:\n            for det in dets:\n                det.position_3d = self._backproject(det.center, depth)\n\n                # Filter by workspace\n                if self.workspace_bounds and det.position_3d is not None:\n                    if not self.workspace_bounds.contains(det.position_3d):\n                        det.in_workspace = False\n                        continue\n                    det.in_workspace = True\n\n        # Track across frames for stability\n        tracked = self.tracker.update(dets)\n\n        return tracked\n\n    def _backproject(self, pixel, depth_image):\n        \"\"\"Convert 2D pixel + depth to 3D point in camera frame.\n\n        This is the INVERSE of projection:\n        X = (u - cx) * Z / fx\n        Y = (v - cy) * Z / fy\n        Z = depth\n        \"\"\"\n        u, v = int(pixel[0]), int(pixel[1])\n\n        # Sample depth with a small window (more robust than single pixel)\n        h, w = depth_image.shape[:2]\n        u = np.clip(u, 2, w - 3)\n        v = np.clip(v, 2, h - 3)\n        patch = depth_image[v-2:v+3, u-2:u+3]\n\n        # Use median to reject outliers (holes, edges)\n        valid = patch[patch > 0]\n        if len(valid) == 0:\n            return None\n        Z = np.median(valid)\n\n        # Convert from depth sensor units (often mm) to meters\n        if Z > 100:  # Likely millimeters\n            Z = Z / 1000.0\n\n        fx, fy = self.K[0, 0], self.K[1, 1]\n        cx, cy = self.K[0, 2], self.K[1, 2]\n\n        X = (u - cx) * Z / fx\n        Y = (v - cy) * Z / fy\n\n        return np.array([X, Y, Z])\n```\n\n### Fiducial Marker Detection (AprilTag / ArUco)\n\n```python\nclass FiducialDetector:\n    \"\"\"Detect and estimate pose of fiducial markers.\n\n    Use cases:\n    - Robot localization (known marker positions)\n    - Object pose estimation (marker attached to object)\n    - Calibration targets\n    - Sim-to-real ground truth\n    \"\"\"\n\n    def __init__(self, K, dist, marker_size_m=0.05,\n                 dictionary=cv2.aruco.DICT_APRILTAG_36h11):\n        self.K = K\n        self.dist = dist\n        self.marker_size = marker_size_m\n        self.aruco_dict = cv2.aruco.getPredefinedDictionary(dictionary)\n        self.params = cv2.aruco.DetectorParameters()\n        self.detector = cv2.aruco.ArucoDetector(self.aruco_dict, self.params)\n\n    def detect(self, image) -> list:\n        \"\"\"Detect markers and estimate their 6DOF poses\"\"\"\n        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n        corners, ids, rejected = self.detector.detectMarkers(gray)\n\n        results = []\n        if ids is not None:\n            for i, marker_id in enumerate(ids.flatten()):\n                # Estimate pose (rotation + translation)\n                rvecs, tvecs, _ = cv2.aruco.estimatePoseSingleMarkers(\n                    corners[i:i+1], self.marker_size, self.K, self.dist)\n\n                R, _ = cv2.Rodrigues(rvecs[0])\n                T = np.eye(4)\n                T[:3, :3] = R\n                T[:3, 3] = tvecs[0].flatten()\n\n                results.append(MarkerDetection(\n                    marker_id=int(marker_id),\n                    corners=corners[i].reshape(4, 2),\n                    pose=T,\n                    distance=np.linalg.norm(tvecs[0]),\n                ))\n\n        return results\n\n    def draw(self, image, detections):\n        \"\"\"Visualize detected markers with axes\"\"\"\n        vis = image.copy()\n        for det in detections:\n            cv2.aruco.drawDetectedMarkers(\n                vis, [det.corners.reshape(1, 4, 2)],\n                np.array([[det.marker_id]]))\n            rvec, _ = cv2.Rodrigues(det.pose[:3, :3])\n            cv2.drawFrameAxes(\n                vis, self.K, self.dist, rvec,\n                det.pose[:3, 3], self.marker_size * 0.5)\n        return vis\n```\n\n### Semantic Segmentation for Manipulation\n\n```python\nclass WorkspaceSegmenter:\n    \"\"\"Segment a robot's workspace into actionable regions.\n\n    Common segmentation targets for manipulation:\n    - Graspable objects vs background\n    - Table / support surface\n    - Robot body (for self-collision avoidance)\n    - Free space vs obstacles\n    \"\"\"\n\n    def __init__(self, model, class_map):\n        self.model = model\n        self.class_map = class_map  # {0: 'background', 1: 'table', 2: 'object', ...}\n\n    def segment(self, rgb) -> SegmentationResult:\n        \"\"\"Run segmentation and extract per-class masks\"\"\"\n        mask = self.model.predict(rgb)  # (H, W) with class IDs\n\n        return SegmentationResult(\n            full_mask=mask,\n            object_mask=(mask == self.class_map['object']),\n            table_mask=(mask == self.class_map['table']),\n            free_space_mask=(mask == self.class_map['free_space']),\n        )\n\n    def get_grasp_candidates(self, rgb, depth, K):\n        \"\"\"Combine segmentation with depth to find graspable regions\"\"\"\n        seg = self.segment(rgb)\n\n        # Find connected components in object mask\n        num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(\n            seg.object_mask.astype(np.uint8))\n\n        candidates = []\n        for i in range(1, num_labels):  # Skip background (0)\n            component_mask = (labels == i)\n            area = stats[i, cv2.CC_STAT_AREA]\n\n            if area < 100:  # Too small, likely noise\n                continue\n\n            # Get median depth of this object\n            obj_depths = depth[component_mask]\n            valid_depths = obj_depths[obj_depths > 0]\n            if len(valid_depths) == 0:\n                continue\n\n            center_px = centroids[i]\n            median_depth = np.median(valid_depths)\n\n            # Backproject center to 3D\n            fx, fy = K[0, 0], K[1, 1]\n            cx, cy = K[0, 2], K[1, 2]\n            X = (center_px[0] - cx) * median_depth / fx\n            Y = (center_px[1] - cy) * median_depth / fy\n\n            candidates.append(GraspCandidate(\n                position_3d=np.array([X, Y, median_depth]),\n                pixel_center=center_px,\n                mask=component_mask,\n                area_pixels=area,\n            ))\n\n        return candidates\n```\n\n## Depth Processing\n\n### Depth Map Filtering and Cleanup\n\n```python\nclass DepthProcessor:\n    \"\"\"Clean up raw depth maps for downstream use.\n\n    Raw depth from sensors is noisy: holes, flying pixels at edges,\n    multi-path interference, and range noise. Always filter before use.\n    \"\"\"\n\n    def __init__(self, depth_scale=0.001, min_depth_m=0.1, max_depth_m=3.0):\n        self.depth_scale = depth_scale  # Raw units to meters\n        self.min_depth = min_depth_m\n        self.max_depth = max_depth_m\n\n    def process(self, raw_depth) -> np.ndarray:\n        \"\"\"Full depth cleanup pipeline\"\"\"\n        depth = raw_depth.astype(np.float32) * self.depth_scale\n\n        # 1. Range filtering\n        depth[(depth < self.min_depth) | (depth > self.max_depth)] = 0\n\n        # 2. Edge filtering — remove flying pixels at depth discontinuities\n        depth = self._remove_flying_pixels(depth)\n\n        # 3. Hole filling — fill small holes via inpainting\n        depth = self._fill_holes(depth, max_hole_size=10)\n\n        # 4. Spatial smoothing — bilateral filter preserves edges\n        depth = self._bilateral_filter(depth)\n\n        return depth\n\n    def _remove_flying_pixels(self, depth, threshold=0.05):\n        \"\"\"Remove pixels at depth discontinuities (flying pixels).\n        These are artifacts where the sensor interpolates between\n        foreground and background.\"\"\"\n        dx = np.abs(np.diff(depth, axis=1, prepend=0))\n        dy = np.abs(np.diff(depth, axis=0, prepend=0))\n        gradient = np.sqrt(dx**2 + dy**2)\n        depth[gradient > threshold] = 0\n        return depth\n\n    def _fill_holes(self, depth, max_hole_size=10):\n        \"\"\"Fill small holes using inpainting. Large holes remain as 0.\"\"\"\n        mask = (depth == 0).astype(np.uint8)\n\n        # Only fill small holes\n        kernel = np.ones((max_hole_size, max_hole_size), np.uint8)\n        small_holes = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)\n        small_holes = small_holes & mask\n\n        if small_holes.any():\n            # Normalize depth to uint16 for inpainting\n            d_norm = (depth * 1000).astype(np.uint16)\n            filled = cv2.inpaint(\n                d_norm, small_holes, max_hole_size, cv2.INPAINT_NS)\n            depth = np.where(small_holes, filled.astype(np.float32) / 1000, depth)\n\n        return depth\n\n    def _bilateral_filter(self, depth, d=5, sigma_color=0.02, sigma_space=5):\n        \"\"\"Bilateral filter: smooths noise while preserving depth edges\"\"\"\n        # Convert to uint16 for cv2 bilateral (doesn't support float)\n        d_uint16 = (depth * 10000).astype(np.uint16)\n        filtered = cv2.bilateralFilter(d_uint16, d,\n                                        sigma_color * 10000, sigma_space)\n        return filtered.astype(np.float32) / 10000\n```\n\n### Depth-to-Point Cloud Conversion\n\n```python\nclass DepthToPointCloud:\n    \"\"\"Convert depth images to organized and unorganized point clouds.\"\"\"\n\n    def __init__(self, K, image_size):\n        self.K = K\n        # Pre-compute pixel coordinate grid (do once, reuse every frame)\n        h, w = image_size[1], image_size[0]\n        u_coords, v_coords = np.meshgrid(np.arange(w), np.arange(h))\n        self.u = u_coords.astype(np.float32)\n        self.v = v_coords.astype(np.float32)\n        self.fx, self.fy = K[0, 0], K[1, 1]\n        self.cx, self.cy = K[0, 2], K[1, 2]\n\n    def convert(self, depth, rgb=None) -> np.ndarray:\n        \"\"\"Convert depth image to point cloud.\n\n        Returns:\n            points: (N, 3) or (N, 6) if rgb provided [x, y, z, r, g, b]\n        \"\"\"\n        # Backproject all pixels in parallel (vectorized)\n        Z = depth.astype(np.float32)\n        X = (self.u - self.cx) * Z / self.fx\n        Y = (self.v - self.cy) * Z / self.fy\n\n        # Stack into (H, W, 3) organized point cloud\n        points = np.stack([X, Y, Z], axis=-1)\n\n        # Filter invalid points\n        valid_mask = Z > 0\n        points_valid = points[valid_mask]\n\n        if rgb is not None:\n            colors = rgb[valid_mask].astype(np.float32) / 255.0\n            return np.hstack([points_valid, colors])\n\n        return points_valid\n\n    def convert_organized(self, depth, rgb=None):\n        \"\"\"Return organized point cloud (H, W, 3) — preserves pixel layout.\n        Invalid points are NaN.\"\"\"\n        Z = depth.astype(np.float32)\n        X = (self.u - self.cx) * Z / self.fx\n        Y = (self.v - self.cy) * Z / self.fy\n\n        organized = np.stack([X, Y, Z], axis=-1)\n        organized[depth == 0] = np.nan\n\n        return organized\n```\n\n### RGBD Alignment\n\n```python\nclass RGBDAligner:\n    \"\"\"Align RGB and depth images from different sensors.\n\n    Most RGBD sensors (RealSense, ZED) provide built-in alignment.\n    Use this when combining separate RGB + depth cameras.\n    \"\"\"\n\n    def __init__(self, K_rgb, K_depth, T_depth_to_rgb, rgb_size, depth_size):\n        self.K_rgb = K_rgb\n        self.K_depth = K_depth\n        self.T = T_depth_to_rgb  # 4x4 transform: depth frame → rgb frame\n        self.rgb_size = rgb_size\n        self.depth_size = depth_size\n\n    def align_depth_to_rgb(self, depth):\n        \"\"\"Reproject depth image into RGB camera's frame.\n\n        For each pixel in depth image:\n        1. Backproject to 3D using depth intrinsics\n        2. Transform to RGB camera frame\n        3. Project to RGB pixel using RGB intrinsics\n        \"\"\"\n        h_d, w_d = depth.shape\n        h_r, w_r = self.rgb_size[1], self.rgb_size[0]\n        aligned = np.zeros((h_r, w_r), dtype=np.float32)\n\n        # Backproject depth pixels to 3D\n        v_d, u_d = np.mgrid[0:h_d, 0:w_d].astype(np.float32)\n        Z = depth.astype(np.float32)\n        valid = Z > 0\n\n        X = (u_d[valid] - self.K_depth[0, 2]) * Z[valid] / self.K_depth[0, 0]\n        Y = (v_d[valid] - self.K_depth[1, 2]) * Z[valid] / self.K_depth[1, 1]\n        pts_3d = np.stack([X, Y, Z[valid], np.ones_like(X)], axis=0)\n\n        # Transform to RGB frame\n        pts_rgb = self.T @ pts_3d\n\n        # Project to RGB pixels\n        u_r = (self.K_rgb[0, 0] * pts_rgb[0] / pts_rgb[2] +\n               self.K_rgb[0, 2]).astype(int)\n        v_r = (self.K_rgb[1, 1] * pts_rgb[1] / pts_rgb[2] +\n               self.K_rgb[1, 2]).astype(int)\n\n        # Fill aligned depth (z-buffer for occlusion handling)\n        in_bounds = (u_r >= 0) & (u_r < w_r) & (v_r >= 0) & (v_r < h_r)\n        for u, v, z in zip(u_r[in_bounds], v_r[in_bounds],\n                           pts_rgb[2][in_bounds]):\n            if aligned[v, u] == 0 or z < aligned[v, u]:\n                aligned[v, u] = z\n\n        return aligned\n```\n\n## Point Cloud Processing\n\n### Filtering and Downsampling\n\n```python\nimport open3d as o3d\n\nclass PointCloudProcessor:\n    \"\"\"Point cloud processing pipeline using Open3D.\n\n    Pipeline order matters:\n    1. Crop to region of interest (reduces data volume first)\n    2. Remove statistical outliers\n    3. Downsample (voxel grid)\n    4. Estimate normals\n    5. Application-specific processing\n    \"\"\"\n\n    def preprocess(self, points, colors=None,\n                   workspace_bounds=None, voxel_size=0.005):\n        \"\"\"Standard preprocessing pipeline\"\"\"\n        pcd = o3d.geometry.PointCloud()\n        pcd.points = o3d.utility.Vector3dVector(points[:, :3])\n        if colors is not None:\n            pcd.colors = o3d.utility.Vector3dVector(colors)\n\n        # 1. Crop to workspace\n        if workspace_bounds:\n            bbox = o3d.geometry.AxisAlignedBoundingBox(\n                min_bound=workspace_bounds['min'],\n                max_bound=workspace_bounds['max'])\n            pcd = pcd.crop(bbox)\n\n        # 2. Statistical outlier removal\n        pcd, inlier_idx = pcd.remove_statistical_outlier(\n            nb_neighbors=20, std_ratio=2.0)\n\n        # 3. Voxel downsampling\n        pcd = pcd.voxel_down_sample(voxel_size=voxel_size)\n\n        # 4. Normal estimation\n        pcd.estimate_normals(\n            search_param=o3d.geometry.KDTreeSearchParamHybrid(\n                radius=voxel_size * 4, max_nn=30))\n        # Orient normals toward camera\n        pcd.orient_normals_towards_camera_location(\n            camera_location=np.array([0, 0, 0]))\n\n        return pcd\n\n    def segment_plane(self, pcd, distance_threshold=0.005,\n                      ransac_n=3, num_iterations=1000):\n        \"\"\"Segment the dominant plane (table / floor) using RANSAC.\n        Returns plane model and inlier/outlier point clouds.\"\"\"\n        plane_model, inliers = pcd.segment_plane(\n            distance_threshold=distance_threshold,\n            ransac_n=ransac_n,\n            num_iterations=num_iterations)\n\n        plane_cloud = pcd.select_by_index(inliers)\n        objects_cloud = pcd.select_by_index(inliers, invert=True)\n\n        a, b, c, d = plane_model\n        print(f\"Plane: {a:.3f}x + {b:.3f}y + {c:.3f}z + {d:.3f} = 0\")\n\n        return plane_model, plane_cloud, objects_cloud\n\n    def cluster_objects(self, pcd, eps=0.02, min_points=50):\n        \"\"\"Cluster point cloud into individual objects using DBSCAN\"\"\"\n        labels = np.array(pcd.cluster_dbscan(\n            eps=eps, min_points=min_points, print_progress=False))\n\n        clusters = []\n        for label_id in range(labels.max() + 1):\n            cluster_mask = labels == label_id\n            cluster_pcd = pcd.select_by_index(\n                np.where(cluster_mask)[0])\n\n            bbox = cluster_pcd.get_axis_aligned_bounding_box()\n            center = bbox.get_center()\n            extent = bbox.get_extent()\n\n            clusters.append(PointCloudCluster(\n                cloud=cluster_pcd,\n                centroid=center,\n                extent=extent,\n                num_points=cluster_mask.sum(),\n                bbox=bbox,\n            ))\n\n        # Sort by size (largest first)\n        clusters.sort(key=lambda c: c.num_points, reverse=True)\n        return clusters\n```\n\n### Point Cloud Registration (ICP)\n\n```python\nclass PointCloudRegistration:\n    \"\"\"Align two point clouds using ICP and its variants.\n\n    Use cases:\n    - Multi-view reconstruction (combine views from different poses)\n    - Object model matching (align scan to CAD model)\n    - LiDAR odometry (align consecutive scans)\n    - Bin picking (match object template to scene)\n    \"\"\"\n\n    def align_icp(self, source, target, initial_transform=np.eye(4),\n                  max_distance=0.05, method='point_to_plane'):\n        \"\"\"Iterative Closest Point alignment.\n\n        Args:\n            source: point cloud to align\n            target: reference point cloud\n            initial_transform: starting guess (CRITICAL for convergence)\n            max_distance: max correspondence distance\n            method: 'point_to_point' or 'point_to_plane'\n        \"\"\"\n        if method == 'point_to_plane':\n            # Point-to-plane converges faster but needs normals\n            if not target.has_normals():\n                target.estimate_normals(\n                    search_param=o3d.geometry.KDTreeSearchParamHybrid(\n                        radius=0.05, max_nn=30))\n            estimation = o3d.pipelines.registration.TransformationEstimationPointToPlane()\n        else:\n            estimation = o3d.pipelines.registration.TransformationEstimationPointToPoint()\n\n        result = o3d.pipelines.registration.registration_icp(\n            source, target, max_distance, initial_transform,\n            estimation,\n            o3d.pipelines.registration.ICPConvergenceCriteria(\n                max_iteration=100,\n                relative_fitness=1e-6,\n                relative_rmse=1e-6))\n\n        print(f\"ICP fitness: {result.fitness:.4f}, \"\n              f\"RMSE: {result.inlier_rmse:.6f}\")\n\n        if result.fitness < 0.3:\n            print(\"WARNING: Low fitness — ICP likely failed. \"\n                  \"Check initial alignment or increase max_distance.\")\n\n        return result.transformation, result\n\n    def align_global(self, source, target, voxel_size=0.01):\n        \"\"\"Global registration (no initial guess needed).\n        Use FPFH features + RANSAC for coarse alignment,\n        then refine with ICP.\"\"\"\n\n        # Downsample for feature computation\n        source_down = source.voxel_down_sample(voxel_size)\n        target_down = target.voxel_down_sample(voxel_size)\n\n        # Compute normals\n        for pcd in [source_down, target_down]:\n            pcd.estimate_normals(\n                o3d.geometry.KDTreeSearchParamHybrid(\n                    radius=voxel_size * 2, max_nn=30))\n\n        # Compute FPFH features\n        source_fpfh = o3d.pipelines.registration.compute_fpfh_feature(\n            source_down,\n            o3d.geometry.KDTreeSearchParamHybrid(\n                radius=voxel_size * 5, max_nn=100))\n        target_fpfh = o3d.pipelines.registration.compute_fpfh_feature(\n            target_down,\n            o3d.geometry.KDTreeSearchParamHybrid(\n                radius=voxel_size * 5, max_nn=100))\n\n        # RANSAC-based global registration\n        coarse = o3d.pipelines.registration.registration_ransac_based_on_feature_matching(\n            source_down, target_down, source_fpfh, target_fpfh,\n            mutual_filter=True,\n            max_correspondence_distance=voxel_size * 2,\n            estimation_method=o3d.pipelines.registration.TransformationEstimationPointToPoint(),\n            ransac_n=3,\n            checkers=[\n                o3d.pipelines.registration.CorrespondenceCheckerBasedOnEdgeLength(0.9),\n                o3d.pipelines.registration.CorrespondenceCheckerBasedOnDistance(voxel_size * 2),\n            ],\n            criteria=o3d.pipelines.registration.RANSACConvergenceCriteria(100000, 0.999))\n\n        # Refine with ICP\n        refined, _ = self.align_icp(\n            source, target,\n            initial_transform=coarse.transformation,\n            max_distance=voxel_size * 1.5,\n            method='point_to_plane')\n\n        return refined\n```\n\n## Multi-Sensor Fusion\n\n### Perception Fusion Architecture\n\n```\n┌──────────────┐    ┌──────────────┐    ┌──────────────┐\n│  RGB Camera  │    │ Depth Sensor │    │    LiDAR     │\n│   30 Hz      │    │   30 Hz      │    │   10 Hz      │\n└──────┬───────┘    └──────┬───────┘    └──────┬───────┘\n       │                   │                   │\n       ▼                   ▼                   ▼\n┌──────────────┐    ┌──────────────┐    ┌──────────────┐\n│  Detection   │    │    Depth     │    │ Point Cloud  │\n│  + Segment   │    │  Filtering   │    │  Processing  │\n└──────┬───────┘    └──────┬───────┘    └──────┬───────┘\n       │                   │                   │\n       └───────────┬───────┘───────────────────┘\n                   ▼\n          ┌────────────────┐\n          │  Fusion Layer  │\n          │  (association  │\n          │   + tracking)  │\n          └────────┬───────┘\n                   ▼\n          ┌────────────────┐\n          │  World Model   │\n          │  (tracked      │\n          │   objects)     │\n          └────────────────┘\n```\n\n### Camera + LiDAR Fusion\n\n```python\nclass CameraLiDARFusion:\n    \"\"\"Project LiDAR points into camera image for cross-modal association.\n\n    Two fusion strategies:\n    1. Early fusion: Project LiDAR → camera, combine at data level\n    2. Late fusion: Detect in each modality, associate results\n\n    Early fusion is simpler and recommended when sensors are calibrated.\n    \"\"\"\n\n    def __init__(self, K, T_lidar_to_camera, image_size):\n        self.K = K\n        self.T = T_lidar_to_camera  # 4x4 transform\n        self.image_size = image_size  # (w, h)\n\n    def project_lidar_to_image(self, lidar_points):\n        \"\"\"Project 3D LiDAR points to 2D image pixels.\n\n        Returns:\n            pixels: (N, 2) pixel coordinates\n            depths: (N,) depth values\n            valid_mask: (N,) bool — True if point projects into image\n        \"\"\"\n        N = len(lidar_points)\n        # Homogeneous coordinates\n        pts_h = np.hstack([lidar_points[:, :3], np.ones((N, 1))])\n\n        # Transform to camera frame\n        pts_cam = (self.T @ pts_h.T).T\n\n        # Points behind camera\n        valid = pts_cam[:, 2] > 0\n        depths = pts_cam[:, 2]\n\n        # Project to pixels\n        pixels = np.zeros((N, 2))\n        pixels[:, 0] = self.K[0, 0] * pts_cam[:, 0] / pts_cam[:, 2] + self.K[0, 2]\n        pixels[:, 1] = self.K[1, 1] * pts_cam[:, 1] / pts_cam[:, 2] + self.K[1, 2]\n\n        # Check image bounds\n        w, h = self.image_size\n        valid &= (pixels[:, 0] >= 0) & (pixels[:, 0] < w)\n        valid &= (pixels[:, 1] >= 0) & (pixels[:, 1] < h)\n\n        return pixels, depths, valid\n\n    def colorize_pointcloud(self, lidar_points, rgb_image):\n        \"\"\"Paint LiDAR points with RGB colors from camera image\"\"\"\n        pixels, depths, valid = self.project_lidar_to_image(lidar_points)\n\n        colors = np.zeros((len(lidar_points), 3), dtype=np.float32)\n        valid_px = pixels[valid].astype(int)\n        colors[valid] = rgb_image[valid_px[:, 1], valid_px[:, 0]] / 255.0\n\n        return colors, valid\n\n    def enrich_detections(self, detections_2d, lidar_points):\n        \"\"\"Add 3D information to 2D camera detections using LiDAR.\n\n        For each 2D bounding box, find LiDAR points that project\n        inside it, and compute the 3D centroid.\n        \"\"\"\n        pixels, depths, valid = self.project_lidar_to_image(lidar_points)\n\n        enriched = []\n        for det in detections_2d:\n            x1, y1, x2, y2 = det.bbox\n\n            # Find LiDAR points inside this bounding box\n            in_box = (valid &\n                      (pixels[:, 0] >= x1) & (pixels[:, 0] <= x2) &\n                      (pixels[:, 1] >= y1) & (pixels[:, 1] <= y2))\n\n            if in_box.sum() > 3:\n                # Use LiDAR points for 3D position (more accurate than depth camera)\n                box_points = lidar_points[in_box]\n\n                # Median is robust to outliers from other objects\n                position_3d = np.median(box_points[:, :3], axis=0)\n                det.position_3d = position_3d\n                det.num_lidar_points = in_box.sum()\n                det.has_3d = True\n            else:\n                det.has_3d = False\n\n            enriched.append(det)\n\n        return enriched\n```\n\n### Multi-Object Tracking\n\n```python\nclass PerceptionTracker:\n    \"\"\"Track detected objects across frames for temporal consistency.\n\n    Why tracking matters in robotics:\n    - Detections flicker (present one frame, absent the next)\n    - Robot needs consistent object IDs for manipulation\n    - Velocity estimation requires temporal association\n    - Occlusion handling prevents phantom disappearances\n    \"\"\"\n\n    def __init__(self, max_age=5, min_hits=3, iou_threshold=0.3):\n        self.max_age = max_age            # Frames before deleting lost track\n        self.min_hits = min_hits          # Frames before confirming track\n        self.iou_threshold = iou_threshold\n        self.tracks = {}\n        self.next_id = 0\n\n    def update(self, detections) -> list:\n        \"\"\"Associate detections with existing tracks.\n\n        Uses Hungarian algorithm for optimal assignment.\n        \"\"\"\n        if not self.tracks:\n            # First frame: create tracks for all detections\n            for det in detections:\n                self._create_track(det)\n            return list(self.tracks.values())\n\n        # Compute cost matrix (IoU for 2D, Euclidean for 3D)\n        track_list = list(self.tracks.values())\n        cost_matrix = self._compute_cost_matrix(track_list, detections)\n\n        # Hungarian assignment\n        from scipy.optimize import linear_sum_assignment\n        row_idx, col_idx = linear_sum_assignment(cost_matrix)\n\n        matched = set()\n        for r, c in zip(row_idx, col_idx):\n            if cost_matrix[r, c] < self.iou_threshold:\n                track_list[r].update(detections[c])\n                matched.add(c)\n\n        # Create new tracks for unmatched detections\n        for i, det in enumerate(detections):\n            if i not in matched:\n                self._create_track(det)\n\n        # Age out old tracks\n        stale = [tid for tid, t in self.tracks.items()\n                 if t.age > self.max_age]\n        for tid in stale:\n            del self.tracks[tid]\n\n        # Return confirmed tracks only\n        return [t for t in self.tracks.values()\n                if t.hits >= self.min_hits]\n\n    def _create_track(self, detection):\n        track = Track(\n            track_id=self.next_id,\n            detection=detection,\n            hits=1, age=0)\n        self.tracks[self.next_id] = track\n        self.next_id += 1\n\n    def _compute_cost_matrix(self, tracks, detections):\n        n_tracks = len(tracks)\n        n_dets = len(detections)\n        cost = np.ones((n_tracks, n_dets))\n\n        for i, track in enumerate(tracks):\n            for j, det in enumerate(detections):\n                if hasattr(det, 'position_3d') and det.position_3d is not None:\n                    # 3D Euclidean distance (preferred)\n                    dist = np.linalg.norm(\n                        track.position_3d - det.position_3d)\n                    cost[i, j] = dist\n                else:\n                    # 2D IoU\n                    cost[i, j] = 1.0 - self._iou(track.bbox, det.bbox)\n\n        return cost\n\n    @staticmethod\n    def _iou(box1, box2):\n        x1 = max(box1[0], box2[0])\n        y1 = max(box1[1], box2[1])\n        x2 = min(box1[2], box2[2])\n        y2 = min(box1[3], box2[3])\n        inter = max(0, x2 - x1) * max(0, y2 - y1)\n        area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])\n        area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])\n        return inter / (area1 + area2 - inter + 1e-6)\n```\n\n## Perception Latency Budget\n\n```\nComponent                   Typical Latency    Budget Target\n───────────────────────────────────────────────────────────────\nSensor capture              1-10 ms            —\nImage transfer (USB3)       2-5 ms             —\nUndistortion                1-2 ms             < 3 ms\nDetection (GPU)             10-50 ms           < 30 ms\nDetection (CPU)             50-500 ms          < 100 ms\nDepth processing            2-5 ms             < 5 ms\nPoint cloud generation      3-10 ms            < 10 ms\nSegmentation (GPU)          15-40 ms           < 30 ms\nTracking update             0.5-2 ms           < 2 ms\n3D pose estimation          5-20 ms            < 15 ms\n───────────────────────────────────────────────────────────────\nTOTAL PIPELINE              ~50-150 ms         < 100 ms target\n\nFor 10 Hz perception: budget = 100 ms per frame\nFor 30 Hz perception: budget = 33 ms per frame\n\nRULE: Perception latency + planning latency < control period\n      If control runs at 100 Hz (10 ms), perception must be\n      pipelined — process frame N while control uses frame N-1.\n```\n\n## Perception Anti-Patterns\n\n### 1. Processing Every Frame When You Don't Need To\n```python\n# ❌ BAD: Running heavy detection at sensor framerate\ndef callback(self, msg):\n    detections = self.heavy_model(msg)  # 100ms on every 30Hz frame\n\n# ✅ GOOD: Decimate or skip frames\ndef callback(self, msg):\n    self.frame_count += 1\n    if self.frame_count % 3 != 0:  # Process every 3rd frame\n        return\n    detections = self.heavy_model(msg)\n```\n\n### 2. Not Validating Depth Values\n```python\n# ❌ BAD: Using raw depth blindly\nz = depth_image[v, u]\npoint_3d = backproject(u, v, z)  # z might be 0 (hole) or 65535 (invalid)\n\n# ✅ GOOD: Always validate depth\nz = depth_image[v, u]\nif z <= 0 or z > max_range:\n    return None\nz_meters = z * depth_scale\n```\n\n### 3. Assuming Cameras Are Calibrated\n```python\n# ❌ BAD: Using default/nominal intrinsics\nK = np.array([[600, 0, 320], [0, 600, 240], [0, 0, 1]])  # Guess\n\n# ✅ GOOD: Load actual calibration\nK = load_calibration(\"camera_serial_12345.yaml\").camera_matrix\n```\n\n### 4. Ignoring Sensor Warmup\n```python\n# ❌ BAD: Using first frames from sensor\ncamera.start()\nframe = camera.capture()  # Often overexposed, auto-exposure not settled\n\n# ✅ GOOD: Discard initial frames\ncamera.start()\nfor _ in range(30):  # Let auto-exposure settle (~1 second at 30Hz)\n    camera.capture()\nframe = camera.capture()  # Now usable\n```\n\n### 5. Single-Pixel Depth Sampling\n```python\n# ❌ BAD: Single pixel is noisy and may be a hole\nz = depth[int(center_y), int(center_x)]\n\n# ✅ GOOD: Sample a neighborhood, use median\npatch = depth[cy-3:cy+4, cx-3:cx+4]\nvalid = patch[patch > 0]\nz = np.median(valid) if len(valid) > 0 else None\n```\n\n### 6. Not Handling Coordinate Frame Transforms\n```python\n# ❌ BAD: Assuming everything is in the same frame\nobject_position = detection.position  # In camera frame? Robot frame? World frame?\nrobot.move_to(object_position)        # Which frame does move_to expect?\n\n# ✅ GOOD: Explicit frame tracking\nobject_in_camera = detection.position_3d       # Camera frame\nobject_in_base = T_base_camera @ np.append(object_in_camera, 1.0)  # Robot frame\nrobot.move_to(object_in_base[:3])\n```\n\n## Production Perception Checklist\n\n1. **Calibrate before deploying** — intrinsics, extrinsics, hand-eye\n2. **Validate calibration** with independent measurements\n3. **Set sensor exposure** appropriately — auto-exposure can cause detection flicker\n4. **Undistort** before any geometric computation\n5. **Filter depth** — remove flying pixels, fill small holes\n6. **Timestamp at capture** — not at processing time\n7. **Track objects** across frames — don't rely on single-frame detections\n8. **Handle sensor failures** — missing frames, degraded depth, overexposure\n9. **Log perception output** — bounding boxes, confidence scores, 3D positions\n10. **Benchmark latency** — measure each pipeline stage, find bottlenecks\n11. **Test with edge cases** — empty scenes, cluttered scenes, reflective surfaces, direct sunlight\n12. **Version your models** — pin detection/segmentation model versions in deployment","tags":["robot","perception","robotics","agent","skills","arpitg1304","agent-skills","ai-coding-assistant","claude-skills"],"capabilities":["skill","source-arpitg1304","skill-robot-perception","topic-agent-skills","topic-ai-coding-assistant","topic-claude-skills","topic-robotics"],"categories":["robotics-agent-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/arpitg1304/robotics-agent-skills/robot-perception","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add arpitg1304/robotics-agent-skills","source_repo":"https://github.com/arpitg1304/robotics-agent-skills","install_from":"skills.sh"}},"qualityScore":"0.544","qualityRationale":"deterministic score 0.54 from registry signals: · indexed on github topic:agent-skills · 189 github stars · SKILL.md body (58,456 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-02T18:54:20.822Z","embedding":null,"createdAt":"2026-04-18T22:05:34.259Z","updatedAt":"2026-05-02T18:54:20.822Z","lastSeenAt":"2026-05-02T18:54:20.822Z","tsv":"'+1':1330,1389,3362 '+3':3174,3178 '+4':6406,6410 '-1':227,472,595,596,673,1155,1269,1398,1399,2052,2925,2936,4166,4239,6179 '-10':171,185,6061,6100 '-100':249 '-120':138 '-150':6129 '-2':3172,3176,6071,6114 '-20':154,202,6122 '-200':199,214 '-3':6404,6408 '-30':217 '-40':6107 '-5':6067,6092 '-50':6078 '-500':6085 '-90':157 '0':371,378,385,386,454,463,466,467,603,609,619,647,659,795,796,809,867,1251,1260,1263,1264,1648,1926,1935,2100,2102,2105,2134,2283,3137,3189,3193,3219,3220,3227,3370,3382,3402,3498,3593,3629,3634,3652,3653,3660,3668,3799,3872,3878,3880,3890,3911,3914,4072,4091,4092,4099,4173,4242,4373,4392,4395,4405,4411,4416,4417,4441,4458,4459,4462,4467,4500,4507,4535,4702,4703,4704,4787,4847,5410,5423,5425,5426,5429,5434,5459,5460,5462,5467,5524,5594,5597,5640,5741,5910,5996,5998,6019,6023,6030,6039,6231,6266,6282,6307,6309,6312,6313,6414,6421 '0.0':932,2842,2845 '0.001':588,1408,3747 '0.005':4606,4714 '0.01':2024,5079 '0.02':3986,4801 '0.025':432,1234 '0.03':1765 '0.05':3292,3846,4948,5011 '0.1':184,2759,3751 '0.2':170 '0.3':153,1683,5053,5716 '0.5':198,213,1680,2989,3445,6113 '0.6':488 '0.9':5204 '0.999':5212 '1':387,391,399,457,470,611,617,639,800,801,814,873,1158,1254,1267,1657,1865,1997,2005,2112,2130,2196,2507,2522,2926,3140,3222,3223,3230,3424,3500,3588,3655,3656,3663,3676,3789,3870,4069,4094,4095,4102,4338,4370,4423,4428,4429,4474,4475,4478,4483,4569,4626,4833,5288,5392,5437,5439,5440,5443,5448,5466,5469,5521,5600,5603,5908,5917,6002,6004,6034,6043,6060,6070,6184,6226,6314,6361,6492 '1.0':707,1649,1721,2061,2094,2736,2852,5982,6480 '1.5':5228 '10':201,216,2690,3826,3901,5251,6077,6102,6135,6165,6574 '100':1933,3210,3606,5033,5151,5166,6087,6131,6139,6163 '1000':2129,3953,3973,4720 '1000.0':2208,3215 '10000':4011,4021,4027 '100000':5211 '100ms':2641,2679,6210 '10mm':1757 '11':593,594,1396,1397,6583 '12':6596 '15':1744,6106,6124 '16sc2':2891 '1e-6':5036,5039,6049 '1f':797,802,811,816 '1khz':236 '1mm':1770 '1μs':2584 '2':461,473,810,815,1156,1258,1270,1875,1907,2556,2723,2726,2927,2937,3155,3159,3165,3228,3231,3396,3426,3502,3661,3664,3800,3884,3886,4100,4103,4345,4412,4424,4465,4468,4481,4484,4528,4579,4648,5130,5195,5208,5298,5361,5409,5414,5421,5432,5435,5446,5449,6008,6010,6028,6037,6066,6091,6116,6241,6501 '2.0':1967,4663 '20':1281,1686,4660 '200':226 '240':6311 '255.0':4190,5525 '2d':1053,1100,1153,2917,2919,2976,3106,5355,5534,5541,5548,5577,5782,5977 '2f':1167 '3':135,149,152,194,212,246,458,626,632,1127,1128,1133,1134,1255,1300,1352,1353,1357,1358,1489,1490,1494,1495,1582,1583,1587,1588,1655,1658,1660,1662,1727,1749,1880,3009,3161,3167,3375,3376,3379,3380,3433,3434,3441,3442,3812,4120,4156,4212,4351,4583,4616,4664,4717,5201,5389,5506,5607,5638,5713,6014,6016,6032,6041,6073,6099,6230,6294,6488,6507 '3.0':3755 '30':137,156,248,485,587,1407,2650,2684,2737,4689,5014,5133,5247,5249,6080,6109,6144,6355 '30hz':173,187,2199,6213,6364 '320':6308 '33':2176,6148 '33ms':2195 '36h11':3296 '3d':346,445,1056,1086,1097,1143,1603,1609,1647,2431,2830,2972,3045,3061,3073,3081,3110,3648,3684,4341,4386,4431,4450,5351,5538,5561,5612,5634,5642,5644,5650,5654,5785,5955,5958,5962,5969,5971,6118,6258,6467,6572 '3f':715,4777,4780,4783,4786 '3rd':6234 '4':197,532,533,612,620,1123,1486,1579,1893,3373,3395,3425,3827,4587,4675,4686,4945,6326,6519 '4f':772,783,788,1023,1030,5045 '4x4':1346,4303,5334 '5':3006,3983,3989,4590,5148,5163,5710,6094,6121,6370,6525 '50':4804,6084,6128 '5mm':1618,1754 '5ms':2637 '6':222,233,428,1231,4123,6424,6534 '600':6306,6310 '65535':6269 '6dof':3327 '6f':5050 '7':6542 '8':6555 '9':224,427,1230,2644,6564 'ab':2291,2377 'absent':5685 'account':1762,2742 'accumul':2739 'accur':5615 'accuraci':578,2586 'achiev':2583 'across':3090,5670,6545 'action':3461 'actual':6318 'adapt':560 'add':5537 'age':3005,5709,5718,5720,5858,5872,5909 'algorithm':5754 'align':2274,4247,4251,4268,4318,4374,4488,4532,4538,4541,4546,4851,4896,4919,4926,4937,4956,4962,5063,5072,5092 'allow':2188 'alpha':2841,2844,2867 'also':1479 'alway':1881,2417,2669,2693,2822,3738,6272 'angl':516,1700 'anti':2616,6182 'anti-pattern':2615,6181 'appli':2896 'applic':4592 'application-specif':4591 'appropri':6511 'apriltag':3250,3295 'architectur':1807,5241 'arduino':2444 'area':1693,3598,3603,3605,3697,3699 'area1':6026,6046 'area2':6035,6047 'arg':2177,2843,4957 'arriv':2069 'artifact':3856 'aruco':3251 'assign':5757,5797,5803,5810 'associ':5262,5284,5305,5699,5747 'assum':6295,6432 'astyp':2928,3915,3954,4012,4188,4398,4469,4485,5513 'attach':3273 'auto':6343,6358,6513 'auto-exposur':6342,6357,6512 'automat':2729 'automot':219 'avail':3050 'avg':2123 'avoid':3481 'ax':1185 'axe':1303,1751,3414 'axi':602,1157,3869,3877,4165,4238,4440,4850,5639 'b':4132,4768,4779 'b2g':1540,1548,1564,1566 'back':2025 'background':2687,3471,3499,3592,3864 'backproject':3100,3645,4133,4339,4382,6259 'bad':517,519,2619,2631,2697,2730,2760,6195,6247,6300,6331,6377,6431 'base':1205,1527,1532,1538,5169,5175,6472,6474,6487 'base-to-gripp':1531 'baselin':1027 'basler/pylon':2589 'bbox':4633,4647,4848,4872,4873 'bbox.get':4855,4858 'behind':5404 'benchmark':6575 'best':5,129,1803 'bgr2gray':551,1370,3333 'bilater':3830,3978,3990,4003 'bin':4929 'black':2848,2857 'blind':6251 'block':1878,2041,2064,2673 'board':425,435,452,455,464,468,499,513,537,663,722,1228,1241,1249,1252,1261,1265,1383,1689,1696 'bodi':3476 'bool':5371 'border':2858 'bottleneck':6582 'bound':1861,2715,2985,2995,2997,3070,4497,4521,4525,4530,4602,4632,4636,4638,4641,4643,4852,5452,5549,5588,6568 'box':4853,5550,5589,5591,5619,5624,5636,6569 'box1':5991,5995,6001,6007,6013,6027,6029,6031,6033 'box2':5992,5997,6003,6009,6015,6036,6038,6040,6042 'budget':6052,6056,6138,6147 'buffer':97,1862,1905,1917,2119,2699,2704,2716,2720,4492 'build':61 'built':4266 'built-in':4265 'bump':1776 'c':4769,4782,4882,5817,5828,5836,5838 'c.num':4883 'cad':4922 'calib':958,960 'calib_left.camera':998 'calib_left.dist':1000 'calib_right.camera':1002 'calib_right.dist':1004 'calibr':36,69,342,402,418,479,490,520,675,684,764,838,885,934,955,973,1015,1020,1046,1070,1176,1221,1360,1434,1501,1595,1672,1676,1711,1715,1743,3276,5316,6298,6319,6322,6493,6503 'calibrationerror':1110 'calibrationresult':818,919 'callback':2767,6203,6221 'cam':336,1640,1654,2477,2519,2553,2600,5398,5408,5413,5428,5431,5442,5445 'cam.close':2612 'cam.gevieee1588.value':2608 'cam.open':2607 'cam2base':1577,1581,1586,1592,1628,1643 'cam2ee':1598,1632 'cam2gripper':1484,1488,1493,1499 'camera':12,56,70,132,146,255,282,304,338,339,344,356,357,364,416,482,687,819,853,897,920,936,938,940,979,1047,1051,1061,1090,1098,1120,1125,1131,1151,1162,1171,1178,1188,1200,1277,1284,1363,1451,1460,1518,1525,1600,1637,1718,1731,1773,1805,1858,1904,1909,1911,2325,2415,2430,2446,2452,2461,2485,2524,2568,2576,2590,2598,2648,2662,2680,3113,4276,4329,4349,4693,4697,4699,5243,5268,5278,5293,5324,5333,5395,5405,5490,5542,5618,6296,6324,6443,6465,6468,6475,6479 'camera-lidar':1161,1717,1730 'camera-target':1283 'camera-to-camera':935 'camera-to-lidar':939,1060 'camera-to-robot':1177 'camera.capture':546,1365,2636,6339,6365,6367 'camera.start':6337,6351 'camera_serial_12345.yaml':6323 'cameralidarfus':5273 'camerastream':1854,2180,2182,2661 'candid':3553,3583,3701 'candidates.append':3681 'capabl':2580 'captur':1835,1845,1866,1879,1896,1940,1953,1959,1969,1984,2020,2114,2124,2620,2654,2682,2787,2790,6059,6537 'case':3263,4906,6587 'caus':6516 'cb':559,563,567 'center':600,608,616,3636,3646,3666,3674,3691,3692,4854,4856,4866,6390,6393 'centroid':3579,3638,4865,5562 'cfg':2492,2502,2536,2543 'cfg.enable':2538 'chain':1614 'chang':1784,1786 'characterist':123 'check':569,718,1113,2272,5061,5450 'checker':5202 'checkerboard':420,448,1077 'checklist':1674,6491 'class':414,944,1182,1831,1853,2140,2407,2811,2942,3253,3453,3490,3496,3515,3523,3710,4035,4249,4558,4894,5272,5665 'clean':3712 'cleanup':3708,3782 'clock':2753 'close':3935 'closest':4954 'cloud':34,66,4032,4045,4116,4159,4209,4548,4561,4735,4754,4760,4792,4794,4807,4862,4890,4899,4960,4966,5256,6097 'cluster':4796,4805,4826,4834,4839,4845,4863,4888 'cluster_mask.sum':4871 'cluster_pcd.get':4849 'clusters.append':4860 'clusters.sort':4879 'clutter':6590 'coars':5091,5172 'coarse.transformation':5223 'coeff':690,823,859,861,903,924,1001,1005 'col':5806,5822 'collect':478,489,1274,1282,1814 'collis':3480 'color':3985,4020,4184,4195,4599,4618,4625,5476,5488,5501,5515,5527 'combin':3558,4272,4911,5294 'common':268,2320,2437,3463 'compat':843 'complet':765 'compon':3571,3594,3621,3695,6053 'comprehens':4 'comput':724,946,2427,2815,2827,2872,2901,4056,5100,5115,5134,5559,5777,5919,6524 'confid':2988,2999,3001,3032,3043,6570 'configur':55,1291,2409,2471 'confirm':5732,5881 'connect':3570 'consecut':4927 'consist':5674,5690 'consum':2709 'contact':237,247 'continu':1391,3085,3611,3635 'control':240,6157,6160,6175 'converg':4973,4996 'convers':4033 'convert':3105,3199,3998,4037,4105,4111,4200 'coord':4074,4076 'coordin':446,2921,4058,5363,5383,6427 'corner':449,508,553,589,592,643,1081,1372,1392,1395,1414,1694,3334,3359,3391,3392 'corners.mean':601 'correct':2821 'correspond':1066,1115,4977,5191 'cost':1769,5778,5790,5811,5825,5920,5933,5972,5979,5987 'could':2390 'count':6225,6229 'counter':1849 'cover':11,501,1690 'coverag':487,495,518,529,599,633,644,645,651,653,658,660 'coverage_map.size':649 'cpu':6083 'creat':5763,5839,5895 'createdevic':2603 'criteria':579,581,584,597,1401,1404,5209 'critic':576,4971 'crop':2846,4570,4627 'cross':5282 'cross-mod':5281 'cv2':405,4002 'cv2.aruco.arucodetector':3313 'cv2.aruco.detectorparameters':3311 'cv2.aruco.dict':3294 'cv2.aruco.drawdetectedmarkers':3421 'cv2.aruco.estimateposesinglemarkers':3358 'cv2.aruco.getpredefineddictionary':3308 'cv2.bilateralfilter':4015 'cv2.calib':558,562,566,1009,1475,1572 'cv2.calibratecamera':696 'cv2.calibratehandeye':1465,1562 'cv2.cc':3601 'cv2.color':550,1369,3332 'cv2.connectedcomponentswithstats':3580 'cv2.cornersubpix':590,1393 'cv2.cv':2890 'cv2.cvtcolor':548,1367,3330 'cv2.drawframeaxes':3435 'cv2.file':849,892 'cv2.filestorage':846,889 'cv2.findchessboardcorners':554,1373 'cv2.getoptimalnewcameramatrix':2862 'cv2.initundistortrectifymap':2882 'cv2.inpaint':3957,3965 'cv2.inter':2910 'cv2.morph':3934 'cv2.morphologyex':3932 'cv2.norm':750,755 'cv2.projectpoints':739,1140,1651 'cv2.remap':2905 'cv2.rodrigues':1117,1418,3368,3431 'cv2.solvepnp':1094,1104,1412 'cv2.stereocalibrate':989 'cv2.term':580,583,1400,1403 'cv2.undistortpoints':2923 'cx':372,381,807,3123,3224,3234,3657,3669,6407,6409 'cy':380,382,812,3128,3225,3239,3658,3677,6403,6405 'd':293,3034,3036,3950,3958,3982,4008,4016,4018,4360,4362,4388,4390,4394,4397,4408,4420,4770,4785 'd.confidence':3041 'd1':982 'd2':984 'daemon':1948 'daniilidi':1482 'data':85,92,1839,1987,2769,2780,2793,2803,4576,5296 'dataclass':1818,1820,1830 'dbscan':4812,4816 'debug':111 'decim':6216 'decod':2778 'decoupl':2653 'dedic':1870 'def':422,477,674,832,881,954,1045,1223,1273,1433,1500,1593,1901,1936,1955,1968,2029,2055,2080,2167,2211,2219,2227,2306,2463,2564,2630,2657,2765,2789,2834,2892,2912,2979,3010,3099,3284,3317,3405,3486,3504,3550,3742,3774,3839,3893,3977,4046,4104,4199,4277,4317,4595,4707,4795,4936,5071,5317,5342,5475,5529,5705,5742,5894,5918,5989,6202,6220 'default':2194,2850 'default/nominal':6302 'degrad':6561 'del':5877 'delay':2775 'delet':5723 'dens':159 'deploy':103,6495,6605 'depth':14,31,59,63,160,179,2181,2426,2509,2514,2548,2978,3014,3048,3052,3064,3103,3108,3132,3142,3169,3201,3556,3561,3614,3619,3620,3624,3626,3628,3633,3641,3644,3671,3679,3689,3702,3704,3715,3721,3745,3749,3753,3758,3765,3767,3770,3772,3778,3781,3784,3792,3793,3795,3796,3798,3807,3809,3811,3820,3822,3834,3836,3838,3844,3850,3868,3876,3887,3892,3897,3913,3945,3952,3967,3974,3976,3981,3996,4010,4029,4038,4107,4112,4203,4241,4254,4275,4283,4285,4290,4297,4300,4305,4315,4319,4323,4325,4336,4343,4383,4489,5244,5254,5364,5366,5411,5473,5493,5564,5617,6089,6244,6250,6253,6274,6276,6292,6374,6388,6402,6527,6562 'depth-to-point':4028 'depth.astype':4140,4221,4401 'depth.shape':4363 'depth_image.shape':3154 'depth_sensor.set':2516 'depthai':296,297 'depthprocessor':3711 'depthtopointcloud':4036 'dequ':1816,1915,1931,2721 'design':1863 'det':3027,3033,3039,3057,3059,3096,3418,5574,5657,5769,5773,5847,5857,5930,5938,5947,5953 'det.bbox':5582,5985 'det.center':3063 'det.corners.reshape':3423 'det.has':5649,5653 'det.in':3082,3086 'det.marker':3428 'det.num':5645 'det.pose':3432,3440 'det.position':3060,3072,3080,5641,5957,5970 'detect':43,78,141,238,723,1359,1385,2638,2676,2939,2945,2951,2963,2971,2977,3011,3018,3025,3249,3255,3318,3322,3409,3411,3420,5253,5301,5531,5533,5543,5576,5668,5680,5745,5748,5767,5771,5795,5835,5844,5850,5898,5905,5906,5924,5932,5950,6075,6082,6198,6206,6237,6517,6554 'detection.position':6441,6466 'detection/segmentation':6601 'detectiontrack':3003 'devic':271,2495,2512,2539,2546 'diagnost':1929,2082 'dict':2084,2171,2232,2312,3307,3315 'dictionari':3293,3309 'diff':2174,2185,2206 'differ':1301,2190,2316,4257,4914 'direct':6594 'disappear':5704 'discard':6348 'discontinu':3808,3851 'dist':693,748,822,824,858,900,902,923,925,1058,1102,1147,1227,1238,2838,2864,2884,3288,3300,5966,5975 'distanc':1739,3399,4712,4741,4743,4947,4975,4978,5026,5067,5192,5225,5964 'distort':689 'divers':1295,1747 'doesn':2740,4004 'domin':4723 'downsampl':4552,4584,4666,5097 'downstream':3718 'draw':3406 'drift':2738 'driven':2748 'driver':305,310,329 'drop':115,1886,2117,2718,2727 'dt':2203,2290,2296,2298,2372,2378,2380 'dtype':534,4380,5507 'dx':3865,3883 'dy':3873,3885 'e':987,1038,1039,2016,2022 'e.g':1076 'earli':5289,5307 'edg':110,510,524,1695,3185,3730,3801,3833,3997,6586 'ee':1341,1344,1350,1355,1622,1625,1629 'effect':2642 'effector':1194,1339,1456 'els':2101,2103,2133,2400,5017,5652,5976,6422 'empti':2045,6588 'enabl':2570,2610 'end':1193,1338,1455 'end-effector':1192,1337,1454 'enrich':5530,5572,5659 'enriched.append':5656 'enter':1334 'enumer':3350,5849,5943,5949 'ep':582,1402,4800,4817,4818 'err':749,757,761 'error':713,729,730,770,779,782,787,826,876,878,931,1043,1148,1165,1166,1615,1679,1726,1753,2021,2028 'errors.append':760 'esp32':2445 'especi':521 'estim':231,2829,2974,3044,3257,3271,3325,3352,4588,4677,5015,5018,5029,5196,5696,6120 'euclidean':5783,5963 'event':254,256,2747 'event-driven':2746 'everi':4063,6186,6212,6233 'everyth':6433 'exampl':2587 'excel':1685 'except':2013,2014 'exist':5750 'expans':1789 'expect':6458 'explicit':6460 'exposur':6344,6359,6510,6514 'extent':4857,4859,4867,4868 'extract':3512 'extrins':38,72,354,933,1714,6497 'extrinsiccalibr':945 'eye':41,75,1175,1196,1207,1435,1448,1477,1502,1515,1574,1742,6500 'eye-in-hand':1195,1447 'eye-to-hand':1206,1514 'f':655,661,709,717,763,767,775,784,790,804,988,1018,1026,1040,1041,1160,1324,1382,1666,1951,2018,2363,4774,5041,5046 'f.timestamp':2361 'fail':1112,5060 'failur':113,6558 'fals':1924,1962,3084,4825,5655 'far':2250 'fast':568,2434,2819,2903 'fast-mov':2433 'faster':4997 'featur':5088,5099,5136,5141,5156,5177 'fiduci':3247,3260 'fiducialdetector':3254 'field':1821 'fill':3814,3815,3894,3902,3918,3956,4487,6531 'filled.astype':3971 'filter':2961,3030,3065,3706,3739,3791,3802,3831,3979,3991,4014,4167,4550,5188,5258,6526 'filtered.astype':4025 'find':974,1059,1083,3563,3569,5551,5583,6581 'first':2277,2513,2547,4578,4878,5761,6333 'fit':5035,5043,5057 'fix':1010,1204,1220,1519,1706,1708 'flag':1008,1103 'flatten':604,3383 'fli':3727,3804,3841,3852,6529 'flicker':2970,5681,6518 'flir/basler':299 'float':168,182,234,1842,4007 'floor':4726 'focal':375,791 'focus':1707,1780 'forc':239 'force/torque':232 'foreground':3862 'forev':2707 'format':844 'found':552,571,1371,1380 'fpfh':5087,5135,5138,5140,5153,5155,5184,5186 'fps':2093,2104,2109,2111,2645,2651,2685,2691 'frame':116,545,549,1364,1368,1833,1848,1885,1985,2007,2038,2058,2068,2113,2116,2143,2193,2197,2237,2254,2260,2264,2269,2271,2287,2305,2332,2338,2342,2347,2349,2369,2383,2635,2640,2666,2675,2696,2733,2754,2757,2777,2782,2879,3091,3114,4064,4306,4308,4331,4350,4445,5396,5671,5684,5721,5730,5762,6142,6151,6172,6177,6187,6214,6219,6235,6334,6338,6350,6366,6428,6438,6444,6446,6448,6454,6461,6469,6482,6546,6553,6560 'frame.data':2678 'frame.timestamp':2292,2373 'framer':6201 'frames.items':2289,2371 'frames.values':2282,2365 'free':3482,3542,3548 'freshest':2695 'fronto':1704 'fronto-parallel':1703 'fs':845,888 'fs.getnode':896,901,907,913 'fs.release':879,917 'fs.write':852,857,862,868,874 'full':1613,1691,3527,3780 'fuse':84 'fusion':2326,5238,5240,5260,5270,5286,5290,5300,5308 'fx':370,373,793,3125,3216,3236,3649,3672 'fy':374,379,798,3130,3217,3241,3650,3680 'g':4131 'g2b':1440,1442,1467,1469,1507,1509,1546,1557,1559 'generat':2486,6098 'geometr':2826,6523 'get':1335,2030,2035,2081,2228,2233,2307,2327,3551,3612 'gige':2460,2574 'global':5073,5080,5170 'good':493,1682,2652,2658,2714,2745,2784,6215,6271,6316,6347,6395,6459 'gpio':2440 'gpu':109,6076,6105 'gradient':3881,3888 'grasp':251,3552 'graspabl':3468,3564 'graspcandid':3682 'gray':547,555,591,1366,1374,1394,3329,3338 'gray.shape':610,618,672 'grid':605,613,621,624,627,630,635,637,4059,4586 'gripper':1462,1534,1536 'gripper-to-bas':1535 'gripper2base':1310,1312,1426,1428 'ground':3282 'grow':2701,2706 'guess':4970,5084,6315 'h':133,147,150,166,180,242,911,929,3152,3166,3520,4065,4081,4154,4210,4359,4364,4376,4393,4510,5341,5385,5454,5470 'h.t':5401 'hand':40,74,1174,1198,1209,1437,1450,1476,1504,1517,1573,1741,6499 'hand-ey':39,73,1173,1740,6498 'handeyecalibr':1183 'handl':4495,5701,6426,6556 'hardwar':108,270,2160,2164,2403,2410,2419,2473 'hardware-sync':2159 'hardware-trigg':2402 'hardwaresyncconfig':2408 'hasattr':5952 'hdr':266 'health':2086 'heavi':6197 'height':870,915 'high':263,711,2386 'high-rat':2385 'high-spe':262 'hit':3008,5712,5727,5729,5893,5907 'hole':3184,3726,3813,3817,3824,3895,3899,3904,3908,3920,3924,3927,3931,3938,3940,3961,3963,3970,6267,6386,6533 'home':1796 'homogen':1347,5382 'horaud':1481 'hungarian':5753,5796 'hz':139,158,203,218,250,5248,5250,5252,6136,6145,6164 'icp':4892,4901,4938,5022,5042,5058,5096,5215,5218 'id':1851,1994,3335,3341,3348,3387,3390,3429,3524,4829,4838,5692,5740,5902,5904,5913,5916 'ids.flatten':3351 'idx':4654,5805,5807,5821,5823 'ignor':2962,6327 'imag':29,480,484,491,506,523,544,565,681,701,719,727,778,828,830,863,869,908,914,926,970,1006,1687,1692,2808,2839,2865,2868,2888,2895,2906,3104,3170,3320,3331,3408,4039,4050,4067,4070,4113,4255,4326,4337,5279,5325,5338,5346,5356,5377,5451,5482,5491,5498,5518,5569,6063,6254,6277 'image.copy':3416 'imageundistort':2812 'img':527,670,679,699,751,964,967,992,995 'img_points.append':642 'implement':76 'import':404,406,412,496,1292,1809,1811,1815,1819,1824,1826,2480,2591,4554,5800 'imu':221,286,2324,2389 'imus':16 'in_box.sum':5606,5648 'includ':507,2773 'increas':5065 'independ':1735,6505 'index':4757,4763,4843 'individu':4809 'indoor':174,188 'industri':300 'inform':5539 'init':423,1224,1902,2168,2835,2980,3285,3487,3743,4047,4278,5318,5706 'initi':4942,4967,5027,5062,5083,5221,6349 'inlier':4653,4738,4758,4764 'inlier/outlier':4733 'inpaint':3819,3906,3949 'input':1323,2454 'insid':5556,5586 'instead':2166 'int':535,607,615,906,912,1847,3135,3138,3388,4470,4486,5514,6389,6392 'integr':2940 'intel':276 'inter':2476,6017,6045,6048 'inter-cam':2475 'interest':4574 'interfer':3734 'interpol':2309,2318,2391,2909,3860 'intrins':37,71,363,401,417,1011,1016,1675,1791,4344,4358,6303,6496 'intrinsiccalibr':415 'invalid':4168,4216,6270 'invers':3118 'invert':1528,4765 'iou':5714,5736,5780,5978,5990 'ip':2569,2596,2599,2606 'iter':586,1105,1406,4719,4750,4752,4953,5032 'j':5946,5974,5981 'jitter':2744 'joint':1798 'k':362,369,393,692,747,794,799,808,813,821,895,922,1057,1101,1146,1226,1236,2837,2860,2863,2883,2887,2934,2983,2993,3287,3298,3557,3651,3654,3659,3662,4049,4053,4090,4093,4098,4101,4280,4282,4293,4296,5320,5328,6304,6320 'k1':981 'k2':983 'keep':2724,2853 'kernel':3921,3936 'key':4880 'khz':228 'known':1608,1738,3266 'l2':756 'label':3576,3577,3590,3596,4813,4828,4836,4837 'labels.max':4832 'lambda':4881 'landscap':119 'larg':3907 'largest':4877 'late':5299 'latenc':114,6051,6055,6154,6156,6576 'latest':1884,2031,2077,2262,2328,2340,2668,2670,2725 'layer':5261 'layout':4215 'least':1299 'left':959,966,994 'len':540,735,758,1779,2001,2121,2396,2398,3191,3631,5379,5503,5927,5931,6419 'length':376,792 'let':6356 'level':5297 'lidar':13,57,191,207,308,314,324,942,1049,1054,1063,1095,1121,1126,1132,1141,1163,1172,1719,1732,4924,5246,5269,5275,5292,5322,5331,5344,5348,5352,5380,5387,5479,5484,5496,5499,5504,5535,5545,5552,5567,5570,5584,5609,5621,5646 'light':165,279 'like':3211,3609,4438,5059 'limit':2626 'linear':2911,5801,5808 'list':2091,2281,3016,3321,5746,5775,5787,5788,5794,5832 'livox':320,325,327 'load':882,884,6317,6321 'local':3265 'locat':4698,4700 'log':6565 'loop':1970 'lost':5724 'low':1768,5056 'low-cost':1767 'luxoni':290 'm':155,172,186,200,215,431,441,476,1031,3291,3305,3750,3754,3768,3773 'manipul':175,1620,1756,2948,3451,3467,5694 'map':32,206,530,634,646,2817,2873,2902,3491,3495,3497,3534,3540,3547,3705,3716 'marker':3248,3261,3267,3272,3289,3303,3323,3347,3386,3389,3412 'markerdetect':3385 'mask':3516,3517,3528,3529,3531,3532,3537,3538,3544,3545,3574,3595,3622,3694,3696,3912,3933,3941,4171,4178,4187,4835,4846,5369 'master':2467,2484,2489,2491,2496,2498,2501,2504,2508,2561 'master.start':2500 'master_cfg.enable':2494 'master_sensor.get':2511 'mat':899,904 'match':2153,4918,4931,5178,5813,5855 'matched.add':5837 'matrix':688,820,854,856,898,921,999,1003,1348,5779,5791,5812,5826,5921,6325 'matter':4568,5677 'max':585,785,1405,2172,2183,2204,3004,3752,3771,3823,3898,3923,3926,3962,4640,4644,4687,4946,4974,4976,5012,5025,5031,5066,5131,5149,5164,5190,5224,5708,5719,5994,6000,6018,6022,6285 'maximum':2187 'maxlen':1916,1932,2722 'may':6383 'mean':780 'measur':1736,6506,6577 'median':3180,3613,3640,3670,3678,3688,5625,6400 'medium':189 'memori':2700 'meter':3207,3763,6290 'method':1474,1571,2439,4949,4979,4988,5197,5229 'metric':2087 'microcontrol':2443 'might':6264 'millimet':3212 'min':486,623,629,652,2360,2987,3000,3007,3748,3766,4635,4639,4802,4819,4821,5711,5728,6006,6012 'misalign':117 'miss':2247,6559 'mm':1766,3205 'modal':89,5283,5304 'mode':2479,2506,2521,2555 'model':105,340,345,2952,2982,2991,3489,3493,4731,4737,4772,4790,4917,4923,5265,6208,6239,6599,6602 'model.detect':2639,2677 'motion':230 'mount':1189,1201,1452 'move':497,662,1211,1293,1325,2435,6456 'ms':2125,2175,2186,2207,6062,6068,6072,6074,6079,6081,6086,6088,6093,6095,6101,6103,6108,6110,6115,6117,6123,6125,6130,6132,6140,6149,6166 'msg':6205,6209,6223,6240 'multi':19,100,2136,2414,2429,3732,4908,5236,5661 'multi-camera':2413,2428 'multi-object':5660 'multi-path':3731 'multi-sensor':18,99,2135,5235 'multi-view':4907 'multipl':87,1215,1290,1699,2145 'must':6168 'mutual':5187 'n':193,196,211,245,4119,4122,4716,4746,4748,5200,5360,5365,5370,5378,5391,5420,5925,5929,5935,5937,6173,6178 'n/a':225,235 'name':1908,1913,1950,2256,2270,2286,2334,2348,2368,2382 'nan':4219 'navig':205,1759 'nb':4658 'nearest':2151 'nearest-neighbor':2150 'need':4999,5085,5689,6192 'neighbor':2152,4659 'neighborhood':6398 'network':2581 'neural':295 'never':1877 'new':2067,5840 'next':5687 'nn':4688,5013,5132,5150,5165 'nois':3610,3737,3993 'noisi':3725,6381 'non':2040,2672 'non-block':2039,2671 'none':703,704,1377,1928,2043,2054,2079,2210,2242,2266,2268,2300,2344,2346,2401,2885,2986,3015,3055,3076,3195,3344,4109,4183,4205,4600,4603,4621,5961,6288,6423 'norm':3951,3959 'normal':564,3944,4589,4676,4679,4691,4695,5000,5004,5006,5116,5125 'np':409,1829 'np.abs':3866,3874 'np.append':1644,6476 'np.arange':4078,4080 'np.array':3243,3427,3685,4701,4814,6305 'np.clip':3157,3163 'np.diff':3867,3875 'np.eye':1122,1485,1578,3372,4944 'np.float32':459,1256,3786,3972,4026,4084,4087,4141,4189,4222,4381,4399,4402,5508 'np.float64':2929 'np.hstack':4192,5386 'np.linalg.inv':1641 'np.linalg.norm':1028,1150,3400,5967 'np.max':786 'np.mean':781,1149,2095,2098,2127 'np.median':3197,3642,5635,6416 'np.meshgrid':4077 'np.mgrid':462,1259,4391 'np.nan':4243 'np.ndarray':1840,3779,4110 'np.ones':3922,4437,5390,5934 'np.sqrt':3882 'np.stack':4161,4234,4432 'np.uint16':3955,4013 'np.uint8':3582,3916,3929 'np.where':3968,4844 'np.zeros':451,531,1248,1659,1661,4375,5419,5502 'ns':3966 'num':483,543,1279,1321,1331,3575,3589,4718,4749,4751,4869 'numpi':407,1827 'o3d':4557 'o3d.geometry.axisalignedboundingbox':4634 'o3d.geometry.kdtreesearchparamhybrid':4682,5009,5126,5144,5159 'o3d.geometry.pointcloud':4611 'o3d.pipelines.registration.compute':5139,5154 'o3d.pipelines.registration.correspondencecheckerbasedondistance':5205 'o3d.pipelines.registration.correspondencecheckerbasedonedgelength':5203 'o3d.pipelines.registration.icpconvergencecriteria':5030 'o3d.pipelines.registration.ransacconvergencecriteria':5210 'o3d.pipelines.registration.registration':5021,5173 'o3d.pipelines.registration.transformationestimationpointtoplane':5016 'o3d.pipelines.registration.transformationestimationpointtopoint':5019,5198 'o3d.utility':4613,4623 'oak':292 'oak-d':291 'obj':525,541,668,677,697,736,740,962,990,3618,3625,3627 'obj_points.append':640 'object':42,77,140,443,2938,2944,3269,3275,3469,3503,3530,3535,3573,3617,4759,4793,4797,4810,4916,4932,5267,5632,5662,5669,5691,6439,6451,6463,6470,6477,6485,6544 'observ':1218 'obstacl':3485 'occlus':4494,5700 'odometri':4925 'often':3204,6340 'old':1887,2728,5860 'one':1888,5683 'oom':2703 'open3d':4555,4565 'opencv':333,842 'opencv-compat':841 'oper':1713,2629 'optim':5756 'option':1825,2033,2062,2231,2311,2517,2551 'order':4567 'organ':4041,4157,4201,4207,4233,4240,4245 'orient':229,1297,1748,4690 'ouster':312,316,319 'ouster-sdk':315 'outdoor':204,220 'outlier':3183,4582,4650,4657,5629 'output':126,2450,6567 'outsid':2964 'overexpos':6341 'overexposur':6563 'p':260,2932 'packag':275 'paint':5483 'parallel':1705,4137 'param':4681,5008 'park':1480 'pass':2613 'passiv':162 'patch':3168,3187,3188,6401,6412,6413 'path':413,836,848,883,891,3733 'pathlib':411 'pattern':421,2617,6183 'pcd':4610,4645,4652,4667,4706,4711,4799,4840,4864,5118 'pcd.cluster':4815 'pcd.colors':4622 'pcd.crop':4646 'pcd.estimate':4678,5124 'pcd.orient':4694 'pcd.points':4612 'pcd.remove':4655 'pcd.segment':4739 'pcd.select':4755,4761,4841 'pcd.voxel':4668 'per':726,777,2878,3514,6141,6150 'per-class':3513 'per-imag':725,776 'percept':3,9,45,104,112,5239,6050,6137,6146,6153,6167,6180,6490,6566 'perceptiontrack':5666 'period':6158 'phantom':5703 'physic':1775 'pick':4930 'pin':6600 'pinhol':343 'pipe':2534,2559 'pipe.start':2542 'pipelin':68,2632,2659,2807,3783,4563,4566,4609,6127,6170,6579 'pixel':365,366,377,574,2849,2855,3102,3107,3136,3139,3151,3690,3698,3728,3805,3842,3848,3853,4057,4135,4214,4334,4355,4384,4454,5357,5359,5362,5417,5418,5422,5436,5458,5461,5465,5468,5472,5492,5511,5563,5593,5596,5599,5602,6373,6379,6530 'plan':6155 'plane':4709,4724,4730,4736,4740,4753,4771,4775,4789,4791,4952,4986,4991,4995,5232 'pnp':1082,1111 'point':33,65,348,358,384,444,526,528,542,669,671,678,680,698,700,737,741,752,806,963,965,968,991,993,996,1052,1055,1067,1087,1096,1099,1114,1142,1152,1602,1610,1635,1638,1646,1652,2914,2916,2920,3111,4031,4044,4115,4118,4158,4160,4169,4174,4176,4193,4197,4208,4217,4547,4560,4598,4615,4734,4803,4806,4820,4822,4870,4884,4889,4898,4950,4955,4959,4965,4980,4982,4984,4989,4993,5230,5255,5276,5349,5353,5374,5381,5388,5403,5480,5485,5500,5505,5536,5553,5571,5585,5610,5620,5622,5637,5647,6096,6257 'point-to-plan':4992 'pointcloud':5477 'pointcloudclust':4861 'pointcloudprocessor':4559 'pointcloudregistr':4895 'points_2d.reshape':2924 'polici':2719 'pose':976,1084,1216,1275,1280,1288,1322,1328,1332,1340,1342,1345,1351,1356,1387,1530,1623,1626,1630,1745,2828,2973,3258,3270,3328,3353,3397,4915,6119 'posit':538,3046,3268,3683,5613,5633,5643,5954,6440,6452,6573 'post':2958,3022 'post-process':2957,3021 'practic':6,1804 'pre':1014,2814,2871,2900,4055 'pre-calibr':1013 'pre-comput':2813,2870,2899,4054 'precis':2456 'prefer':2418,5965 'prepar':442 'prepend':3871,3879 'preprocess':4596,4608 'present':5682 'preserv':3832,3995,4213 'press':1333 'prevent':5702 'princip':383,805 'principl':1864 'print':654,708,762,766,774,789,803,1017,1025,1159,1381,1665,2017,4773,4823,5040,5054 'process':30,67,1876,1899,2622,2656,2688,2763,2766,2806,2959,3023,3703,3775,4549,4562,4594,5259,6090,6171,6185,6232,6540 'produc':2713 'product':6489 'profil':2541 'profile.get':2545 'progress':4824 'project':388,738,754,759,1139,1606,1633,1650,1668,3120,4352,4451,5274,5291,5343,5350,5375,5415,5555 'projected.flatten':1669,1671 'projected.reshape':1154 'proper':94 'protocol':2458 'provid':4126,4264 'ptp':2455,2566,2571,2579,2611 'ptp-capabl':2578 'pts':4430,4446,4449,4460,4463,4476,4479,4526,5384,5397,5400,5407,5412,5427,5430,5441,5444 'publish':2781 'puls':2441 'pure':1304 'px':716,773,1024,1168,1681,1684,1722,1728,3637,3667,3675,3693,5510,5520,5523 'py':2594 'py.cdeviceinfo':2604 'py.instantcamera':2601 'py.tlfactory.getinstance':2602 'pypylon.pylon':2592 'pyrealsense2':280,2481 'pyspin/pypylon':302 'python':403,943,1181,1808,2139,2406,2618,2810,2941,3252,3452,3709,4034,4248,4553,4893,5271,5664,6194,6246,6299,6330,6376,6430 'pyze':287 'qualiti':252,720,1673 'queue':2774 'queue.put':2801 'r':352,394,985,1034,1035,1116,1129,1309,1313,1417,1421,1425,1429,1439,1443,1463,1466,1470,1491,1506,1510,1539,1543,1545,1552,1556,1560,1563,1567,1584,3367,3377,4130,4365,4367,4377,4379,4456,4472,4499,4502,4504,4506,4509,4511,4519,4523,5816,5827,5833 'r.t':1541,1549 'r_gripper2base.append':1349 'r_target2cam.append':1420 'radius':4683,5010,5127,5145,5160 'rais':1109 'rang':127,177,190,734,1320,3587,3736,3790,4831,6286,6354 'ransac':4715,4728,4745,4747,5089,5168,5174,5199 'ransac-bas':5167 'rate':128,1874,2317,2387,2643 'ratio':4662 'raw':1978,1988,2768,2779,2792,2802,3024,3026,3038,3714,3720,3760,3777,6249 'raw_depth.astype':3785 're':1795 're-hom':1794 'reach':2966 'read':894 'real':910,916,1891,3281 'real-tim':1890 'realsens':277,2465,2472,4262 'realsense2':281 'recalibr':1771,1800 'receiv':2525 'recent':2037 'recommend':2851,5312 'reconstruct':2432,2831,4910 'reduc':4575 'ref':2279,2293,2358,2374 'refer':2357,4964 'refin':575,5094,5213,5216,5234 'reflect':1079,6592 'regardless':2646 'region':503,666,3462,3565,4572 'registr':4891,5081,5171 'reject':2969,3182,3336 'relat':975,1088,5034,5037 'reli':6549 'remain':3909 'remount':1778 'remov':3803,3840,3847,4580,4651,6528 'repeat':1761 'reproject':712,728,769,1138,1164,1678,1725,4324 'requir':1210,2577,5697 'reshap':1656,2935,3394 'result':835,2366,2381,2394,2397,3339,3404,5020,5070,5306 'result.camera':855 'result.dist':860 'result.fitness':5044,5052 'result.image':865,871 'result.inlier':5048 'result.rms':877 'result.transformation':5069 'results.append':3384 'ret':691,706,714,771,827,980,1022,1044,1409 'return':667,686,817,918,1032,1169,1424,1497,1590,1670,2042,2050,2053,2075,2078,2106,2241,2267,2299,2304,2345,2393,2560,2904,2922,3097,3194,3242,3403,3446,3525,3700,3837,3891,3975,4024,4117,4191,4196,4206,4244,4545,4705,4729,4788,4887,5068,5233,5358,5471,5526,5658,5774,5880,5884,5986,6044,6236,6287 'reus':4062 'revers':4885 'rgb':28,62,131,169,183,301,332,2179,2805,3013,3029,3507,3519,3555,3568,4108,4125,4180,4185,4204,4252,4274,4281,4287,4288,4294,4302,4307,4311,4321,4328,4348,4354,4357,4444,4447,4453,4461,4464,4477,4480,4527,5242,5481,5487,5517 'rgbd':4246,4260 'rgbdalign':4250 'rig':102 'right':961,969,997,2797 'rms':768,825,875,930,1021,1042,1677,1720 'rmse':5038,5047,5049 'robot':2,8,44,83,107,1180,1191,1213,1278,1287,1296,1326,1336,1529,1599,1760,1792,2947,2955,2965,3020,3264,3457,3475,5679,5688,6445,6481 'robot-percept':1 'robot.get':1343,1624 'robot.move':6449,6483 'robotics-specif':2954 'robotobjectdetector':2943 'robust':3148,5627 'ros':298 'ros2':274,318,328 'rotat':1302,1750,3354 'round':2110,2126 'row':5804,5820 'rs':2483 'rs.config':2493,2537 'rs.option.inter':2518,2552 'rs.pipeline':2490,2535 'rule':6152 'ruler':1737 'run':683,1867,3017,3509,6161,6196 'runtim':2820 'rvec':694,743,1092,1118,1144,1410,1419,3356,3369,3430,3439 's.start':2218 's.stop':2226 'safe':1857 'sampl':3141,4670,5105,5112,6375,6396 'save':833,837 'scale':3746,3757,3759,3788,6293 'scan':4920,4928 'scene':267,2436,4935,6589,6591 'schedul':2776 'scipy.optimize':5799 'score':6571 'sdk':317,326 'sdk/driver':273 'search':4680,5007 'second':6362 'seg':3566 'seg.object_mask.astype':3581 'segment':79,3449,3455,3464,3505,3510,3559,4708,4721,5257,6104 'segmentationresult':3508,3526 'self':424,481,676,834,957,1050,1225,1276,1438,1505,1596,1903,1938,1957,1971,2032,2059,2083,2169,2213,2221,2230,2310,2836,2894,2915,2981,3012,3101,3286,3319,3407,3479,3488,3506,3554,3744,3776,3843,3896,3980,4048,4106,4202,4279,4322,4597,4710,4798,4939,5074,5319,5347,5478,5532,5707,5744,5897,5922,6204,6222 'self-collis':3478 'self._backproject':3062 'self._bilateral_filter':3835 'self._buffer':1914,2002,2049,2051,2122 'self._buffer.append':2006 'self._buffer.maxlen':2003 'self._capture_loop':1947 'self._capture_times':1930,2089,2092,2132 'self._capture_times.append':2009 'self._compute_cost_matrix':5792 'self._create_track':5772,5856 'self._drop_count':1934,2004,2118 'self._fill_holes':3821 'self._iou':5983 'self._lock':1919,1999,2047 'self._new_frame':1921 'self._new_frame.clear':2070 'self._new_frame.set':2008 'self._new_frame.wait':2072 'self._remove_flying_pixels':3810 'self._running':1923,1942,1961,1973 'self._sequence':1925,1992,1996,2115 'self._synced_callback':2209 'self._thread':1927,1944,1964 'self._thread.join':1965 'self._thread.start':1954 'self.align':5217 'self.aruco':3306,3314 'self.board':433,556,1239,1375 'self.camera':1910 'self.camera.capture':1979 'self.class':3494,3533,3539,3546 'self.cx':4096,4144,4225 'self.cy':4097,4149,4230 'self.depth':3756,3787,4313 'self.detector':3312 'self.detector.detectmarkers':3337 'self.dist':1237,1416,1664,2931,3299,3366,3438 'self.frame':6224,6228 'self.fx':4088,4146,4227 'self.fy':4089,4151,4232 'self.get':2076 'self.heavy':6207,6238 'self.image':5336,5455 'self.iou':5734,5829 'self.k':1235,1415,1663,2930,2992,3218,3221,3226,3229,3297,3365,3437,4052,5327,5424,5433,5438,5447 'self.k_depth':4295,4410,4415,4422,4427 'self.k_rgb':4292,4457,4466,4473,4482 'self.map1':2880,2907 'self.map2':2881,2908 'self.marker':3301,3363,3443 'self.max':2202,2297,2379,3769,3797,5717,5871 'self.min':2998,3042,3764,3794,5726,5892 'self.model':2990,3028,3492 'self.model.predict':3518 'self.name':1912,1952,1995,2019,2108 'self.new':2859,2886,2933 'self.next':5739,5903,5912,5915 'self.objp':450,460,641,1247,1257,1413 'self.params':3310,3316 'self.project':5495,5566 'self.rgb':4309,4368,4371 'self.roi':2861 'self.segment':3567 'self.sensors':2200,2399 'self.sensors.items':2259,2337 'self.sensors.values':2217,2225 'self.square':437,1243 'self.t':4298,4448,5329,5399 'self.tracker':3002 'self.tracker.update':3095 'self.tracks':5738,5760,5878,5911 'self.tracks.items':5868 'self.tracks.values':5776,5789,5889 'self.u':4082,4143,4224 'self.v':4085,4148,4229 'self.workspace':2994,3069 'self.workspace_bounds.contains':3079 'semant':3448 'sensor':15,20,35,60,88,91,101,118,120,124,269,950,1075,1801,1850,1873,1993,2107,2137,2146,2162,2170,2178,2201,2240,2245,2278,2314,2331,2353,2388,2499,2510,2515,2544,2549,3202,3723,3859,4258,4261,5237,5245,5314,6058,6200,6328,6336,6509,6557 'sensor.read':2794 'sensor.set':2550 'separ':4273 'sequenc':1846,1991 'serial':2468,2470,2497,2530,2533,2540 'servo':144,2833 'set':52,2503,5814,6508 'setipaddress':2605 'settl':6346,6360 'setup':21,2464,2565 'share':952 'shift':1790 'short':176 'show':2856 'sigma':3984,3987,4019,4022 'signal':2488,2527 'signific':1787 'sim':3279 'sim-to-r':3278 'simpler':5310 'singl':3150,6372,6378,6552 'single-fram':6551 'single-pixel':6371 'size':426,430,434,436,438,440,453,456,465,469,475,557,682,702,829,831,866,872,927,971,1007,1229,1233,1240,1242,1244,1246,1250,1253,1262,1266,1272,1376,1906,1918,2120,2840,2866,2869,2889,3290,3302,3304,3364,3444,3825,3900,3925,3928,3964,4051,4068,4071,4289,4291,4310,4312,4314,4316,4369,4372,4605,4672,4674,4685,4876,5078,5107,5114,5129,5147,5162,5194,5207,5227,5326,5337,5339,5456 'skill':24,46,51 'skill-robot-perception' 'skip':1390,3591,6218 'slave':2469,2523,2528,2532,2557,2562 'slaves.append':2558 'sleep':2731 'slower':2711 'slowest':2352,2628 'small':3145,3608,3816,3903,3919,3930,3937,3939,3960,3969,6532 'small_holes.any':3943 'smooth':3829,3992 'softwar':2422 'solid':209,322 'solid-st':208,321 'solv':1184,1457,1522 'sort':4874 'sourc':2800,4940,4958,5023,5075,5101,5120,5137,5142,5179,5183,5219 'source-arpitg1304' 'source.voxel':5103 'space':3483,3543,3549,3988,4023 'spatial':494,3828 'specif':2956,4593 'speed':264 'spin':192,307,313,2750 'spinnak':303 'squar':429,439,474,1232,1245,1271 'st':210 'stabil':2967,3093 'stack':4152 'stage':6580 'stale':5862,5876 'stampedfram':1832,1986,2034,2063 'standard':4607 'start':1937,1939,1975,2012,2212,4969 'stat':3578,3599,3602 'state':323 'staticmethod':880,2462,2563,5988 'statist':4581,4649,4656 'std':4661 'stereo':145,163,285,294,956,972,1019,1716,1724,2425 'stereocalibr':1033 'stereolab':283 'still':2681 'stop':1956,1958,2220 'storag':850,893 'str':847,890,1852 'strategi':5287 'stream':90,1802,1806,1859,2085,2257,2335,2614,2660 'stream.get':2261,2339,2667 'stream.start':2663 'stream.wait':2755 'structur':164,278 'sub':573 'sub-pixel':572 'success':1091,1108 'suffici':1308 'sum':648,5802,5809 'sunlight':6595 'support':3473,4006 'surfac':3474,6593 'switch':2582 'sync':2161,2192,2229,2253,2303,2308,2420,2423,2449,2466,2474,2478,2487,2520,2526,2554,2567,2585 'syncedmultisensor':2141 'synchron':98,1838,2138,2142,2236,2405,2416,2572 'system':10 't.age':5870 't.flatten':1496,1589 't.hits':5891 't.reshape':471,1268 't2c':1444,1446,1471,1473,1511,1513,1568,1570 't_gripper2base.append':1354 't_target2cam.append':1422 'tabl':3472,3501,3536,3541,4725 'tactil':241 'tape':1080 'target':953,1071,1222,1285,1361,1601,1645,1946,3277,3465,4941,4963,5024,5076,5108,5122,5152,5157,5181,5185,5220,6057,6133 'target.estimate':5005 'target.has':5003 'target.voxel':5110 'target2cam':1314,1316,1430,1432 'task':1621 'temperatur':1785 'templat':4933 'tempor':5673,5698 'test':6584 'textur':253 'thermal':1788 'thread':95,1810,1856,1871,1941,1960,2625,2791 'thread-saf':1855 'threading.event':1922 'threading.lock':1920 'threading.thread':1945 'thresh':561 'threshold':3845,3889,4713,4742,4744,5715,5735,5737,5830 'tid':5863,5865,5874,5879 'tilt':511,1697 'time':1812,1892,1900,2090,2096,2099,2128,2156,2173,2184,2189,2205,2235,2273,2280,2294,2359,2375,2457,2734,2764,2788,6541 'time-synchron':2234 'time.monotonic':1843,1976,1981,2010,2796 'time.sleep':2023,2735 'time.time':2771 'timeout':1966,2060,2073,2074,2758 'timestamp':1836,1841,1894,1980,1982,1989,1990,2148,2284,2321,2355,2761,2770,2783,2785,2795,2798,2804,6535 'tof':178 'toler':2157 'topic-agent-skills' 'topic-ai-coding-assistant' 'topic-claude-skills' 'topic-robotics' 'total':6126 'toward':4692,4696 'track':81,142,265,536,598,2968,3089,3094,3098,5263,5266,5663,5667,5676,5725,5733,5751,5764,5786,5793,5831,5841,5861,5882,5896,5899,5900,5901,5914,5923,5926,5928,5936,5941,5944,6111,6462,6543 'track.bbox':5984 'track.position':5968 'transfer':6064 'transform':947,1064,4304,4346,4442,4943,4968,5028,5222,5335,5393,6429 'translat':1305,3355 'tri':1977 'trigger':2165,2404,2411,2438,2453 'true':1943,1949,2609,2634,2665,3088,4766,4886,5189,5372,5651 'truth':3283 'tsai':1478,1575 'tvec':695,745,1093,1145,1411,3357,3381,3401 'tvec.flatten':1135,1423 'two':949,978,4897,5285 'type':121,125,272,1823 'typic':6054 'u':367,389,3122,3133,3156,3158,3175,3177,3233,4073,4389,4407,4455,4498,4501,4513,4518,4534,4540,4543,6256,6260,6279 'u_coords.astype':4083 'uint16':3947,4000,4009,4017 'uint8':136 'unbound':2698 'uncov':665 'undistort':2809,2816,2823,2893,2897,2913,2918,6069,6520 'unit':3203,3761 'unmatch':5843 'unorgan':4043 'updat':5743,5834,6112 'ur5':1764 'usabl':6369 'usb':330,335 'usb3':6065 'use':22,49,419,951,1012,1065,1068,1882,2149,2163,2322,2350,2898,3179,3262,3719,3741,3905,4269,4342,4356,4564,4727,4811,4900,4905,5086,5544,5608,5752,6176,6248,6301,6332,6399 'v':368,390,3127,3134,3162,3164,3171,3173,3238,4075,4387,4419,4471,4505,4508,4514,4522,4533,4539,4542,6255,6261,6278 'v4l2':337 'v_coords.astype':4086 'valid':3186,3192,3198,3623,3632,3643,4170,4175,4177,4186,4194,4198,4403,4409,4414,4421,4426,4436,5368,5406,5457,5464,5474,5494,5509,5512,5516,5519,5522,5528,5565,5592,6243,6273,6411,6417,6420,6502 'valu':5367,6245 'variant':4904 'various':515 'vector':4138 'vector3dvector':4614,4624 'veloc':5695 'velodyn':306,309,311 'verif':1667,1752 'verifi':1136,1594,1604,1733 'version':6597,6603 'via':3818 'videocaptur':334 'view':4909,4912 'vis':3415,3422,3436,3447 'visibl':1072 'vision':2575 'visual':143,2832,3410 'volum':4577 'voxel':4585,4604,4665,4671,4673,4684,5077,5106,5113,5128,5146,5161,5193,5206,5226 'vs':3470,3484 'w':134,148,151,167,181,243,905,928,3153,3160,3521,4066,4079,4155,4211,4361,4366,4378,4396,4503,5340,5453,5463 'wait':2056 'wall':2752 'warmup':6329 'warn':656,710,5055 'webcam':331 'width':864,909 'window':3146 'within':2154 'work':26 'workspac':1521,2960,2984,2996,3067,3083,3087,3459,4601,4629,4631,4637,4642 'workspacesegment':3454 'world':347,355,1634,5264,6447 'wrap':2949 'wrapper':289 'write':851 'wrong':2772 'x':257,349,396,606,622,625,638,3121,3232,3244,3665,3686,4127,4142,4162,4223,4235,4406,4433,4439,4778,6394 'x1':5578,5595,5993,6021 'x2':5580,5598,6005,6020 'xb':1186 'xc':359 'y':258,350,397,614,628,631,636,3126,3237,3245,3673,3687,4128,4147,4163,4228,4236,4418,4434,4781,6391 'y1':5579,5601,5999,6025 'y2':5581,5604,6011,6024 'yaml':840,887 'yc':360 'z':351,398,3124,3129,3131,3196,3209,3213,3214,3235,3240,3246,4129,4139,4145,4150,4164,4172,4220,4226,4231,4237,4400,4404,4413,4425,4435,4491,4515,4537,4544,4784,6252,6262,6263,6275,6281,6284,6289,6291,6387,6415 'z-buffer':4490 'zc':361 'zed':284,288,4263 'zip':1555,4517,5819 'zoom':1709,1782 'μs':261","prices":[{"id":"a105d8d5-0ea7-439b-be1c-e70df9669647","listingId":"140d4839-13a6-4832-95c3-22379efbdfe0","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"arpitg1304","category":"robotics-agent-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T22:05:34.259Z"}],"sources":[{"listingId":"140d4839-13a6-4832-95c3-22379efbdfe0","source":"github","sourceId":"arpitg1304/robotics-agent-skills/robot-perception","sourceUrl":"https://github.com/arpitg1304/robotics-agent-skills/tree/main/skills/robot-perception","isPrimary":false,"firstSeenAt":"2026-04-18T22:05:34.259Z","lastSeenAt":"2026-05-02T18:54:20.822Z"}],"details":{"listingId":"140d4839-13a6-4832-95c3-22379efbdfe0","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"arpitg1304","slug":"robot-perception","github":{"repo":"arpitg1304/robotics-agent-skills","stars":189,"topics":["agent-skills","ai-coding-assistant","claude-skills","robotics"],"license":"apache-2.0","html_url":"https://github.com/arpitg1304/robotics-agent-skills","pushed_at":"2026-03-25T03:44:12Z","description":"Agent skills that make AI coding assistants write production-grade robotics software. ROS1, ROS2, design patterns, SOLID principles, and testing — for Claude Code, Cursor, Copilot, and any SKILL.md-compatible agent.","skill_md_sha":"31e9657c8a37b74aeec59c368d039a6674f47860","skill_md_path":"skills/robot-perception/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/arpitg1304/robotics-agent-skills/tree/main/skills/robot-perception"},"layout":"multi","source":"github","category":"robotics-agent-skills","frontmatter":{"name":"robot-perception","description":"Comprehensive best practices for robot perception systems covering cameras, LiDARs, depth sensors, IMUs, and multi-sensor setups. Use this skill when working with RGB image processing, depth maps, point clouds, sensor calibration (intrinsic, extrinsic, hand-eye), object detection, semantic segmentation, 3D reconstruction, visual servoing, or perception pipeline optimization. Trigger whenever the user mentions OpenCV, Open3D, PCL, RealSense, ZED, OAK-D, camera calibration, AprilTags, ArUco markers, stereo vision, RGBD, point cloud filtering, ICP registration, coordinate transforms, camera intrinsics, distortion correction, image undistortion, sensor streaming, frame synchronization, or any computer vision task in a robotics context. Also covers multi-camera rigs, time synchronization across sensors, perception latency budgets, and production deployment of perception pipelines."},"skills_sh_url":"https://skills.sh/arpitg1304/robotics-agent-skills/robot-perception"},"updatedAt":"2026-05-02T18:54:20.822Z"}}