RoboticsXR: Extended Reality for Robotics Visual Navigation

Virtual Environments and Extended Reality (XR) have transformed artificial intelligence and robotics research by enabling realistic simulations for training and testing. XR enables the seamless integration of digital and physical spaces, offering new opportunities for training and testing autonomous systems. In this work, we propose RoboticsXR, an XR-based framework where a robot operates in a physical lab while images are captured from a virtual environment. This approach fuses real-world motion with synthetic visual data to develop and validate visual navigation algorithms efficiently. RoboticsXR feasibility is assessed using a convolutional neural network trained on synthetic images from NVIDIA Isaac. Performance is compared across different hardware platforms, and using virtual and real camera images. Real-time performance is evaluated for a UAV moving in a confined test area, while virtual images from a synthetic mountain environment are shared with the robotic platform. Results highlight RoboticsXR's potential to extend laboratory testing capabilities, reduce costs, and improve the reliability of vision-based navigation in autonomous robotics systems.