SpiderBot Bridging the Sim-to-Real Gap with Reinforcement Learning

SpiderBot is a 12 DOF quadruped robot controlled entirely by a neural network locomotion policy trained using DRL. The policy was trained in NVIDIA Isaac Lab for 50,000+ PPO iterations, leveraging massive parallel simulations.

The primary research objective was addressing the 'Sim-to-Real Gap': successfully exporting a rigid-body dynamics policy trained in an idealized simulation and deploying it into the real world using Jetson Orin Nano. This required building a highly deterministic 50 Hz control loop capable of real-time state estimation and actuator compensation.

Transferring locomotion policies to physical hardware introduces non-trivial nonlinearities: • Sensor Reality: Simulation provides set state matrices. Real hardware relies on noisy, drifting MEMS IMU sensors requiring active filtering. • Actuator Dynamics: Simulated joints possess infinite bandwidth. Physical servos exhibit bandwidth limits, torque saturation, deadbands, backlash, and voltage sag. • Real-Time Constraints: Inference must execute within a strict 20ms time window alongside sensor acquisition and I2C actuation, necessitating extreme computational efficiency.

This project established a robust, deterministic software architecture to mitigate these discrepancies.

The electromechanical foundation was engineered for modularity and decoupled power delivery:

• Compute Node: NVIDIA Jetson Orin Nano Developer Kit (8GB) executing accelerated ONNX inference. • Actuation: 12× high-torque digital servos driven by a 12-bit PCA9685 PWM controller. • Proprioception: MPU-6050 6-axis IMU for high-frequency attitude estimation. • Power Electronics: A strictly decoupled distribution network utilizing high-capacity buck converters to isolate the compute node from high-current actuator transients. • Safety Engineering: Hardware-level E-stop interrupts actuator power while maintaining compute node telemetry for post-fault analysis.

I engineered a hardened, multi-threaded 50 Hz control loop that strictly orchestrates the sense-plan-act cycle to prevent temporal jitter:

1. State Acquisition: High-speed I2C polling of the MPU-6050 IMU. 2. Sensor Fusion: Execution of a custom NumPy-based Madgwick AHRS (Attitude and Heading Reference System) filter to derive an accurate gravity vector from raw accelerometer/gyroscope data. 3. Observation Assembly: Mapping proprioceptive and user-command data into the strict 235-dimensional float32 tensor required by the model. 4. Edge Inference: Millisecond-latency execution of the ONNX-exported neural network via GPU-accelerated onnxruntime. 5. Actuation Dispatch: Translation of normalized joint space outputs to calibrated PWM pulses, dispatched synchronously to the actuator controller.

1. Recurrent Policy Execution The exported ML policy utilized a LEAPP-format graph featuring multiple independently named input tensors and a stateful recurrent feedback loop. I developed a custom inference engine that natively interfaces with the LEAPP structure, deterministically mapping tensors and managing the hidden state across execution frames without breaking the real-time budget.

2. Real-Time Sensor Fusion To avoid the overhead of heavy scientific libraries on the edge compute node, I implemented a Madgwick Gradient-Descent AHRS Filter in pure Python/NumPy. This highly optimized implementation handles dynamic gyroscope bias calibration and computes the orientation quaternion in microseconds, providing the essential gravity vector for the locomotion policy.

3. Domain Randomization & Graceful Degradation The original policy was trained with a 187-dimensional exteroceptive LIDAR height-map. Because I am yet to implement the pyhsical LIDAR, I exploited the domain randomization used during training, safely zeroing out the exteroceptive vector. This allowed the robot to successfully exhibit flat-ground locomotion immediately, validating the proprioceptive policy transfer.

This project serves as a comprehensive validation of modern reinforcement learning for physical robotics. By engineering a system with explicit temporal bounds, robust I/O error handling, and rigorous sensor fusion, I successfully translated a neural network into real-world quadrupedal locomotion. It demonstrates a deep understanding of rigid body dynamics, state estimation, and edge AI deployment crucial for advanced robotics research and autonomous systems engineering.