Why RGB Cameras are the Superior Choice for Hand Tracking?

The moment we all have been waiting for is here… Apple has released a headset and the big question is what happens now? As soon as the headset was out, I asked our CTO, Shahrouz Yousefi to share his thoughts. See below a very good summary of what this means to us:

As human beings, our perception of the world relies on the richness of color. When transitioning to virtual or mixed reality environments, we desire a realistic representation of our surroundings. This includes the ability to see our own hands and interact with a combination of physical and virtual objects. While infrared cameras have been utilized for hand tracking in commercial products for many years, they lack a fundamental capability: the ability to detect and visualize real hands in the virtual world due to the absence of color information.

At ManoMotion, we recognized this essential requirement from the early stages and responded by developing our standard hand tracking SDK specifically designed for RGB cameras. This innovative solution not only offers hand skeleton and gesture control features, but it also allows for the visualization of real hands in virtual and mixed reality spaces. The recent introduction of the Apple Vision Pro headset, which incorporates similar cameras for hand tracking and the ability to display real hands in virtual space, aligns perfectly with our vision of the ultimate hand tracking sensor.

Considering that the majority of modern devices are equipped with at least one RGB camera, ManoMotion capitalizes on these existing sensors to offer extensive interaction possibilities, eliminating the need for additional hardware, processing units, or extra controllers. This presents a remarkable opportunity to expand the usability of current devices through a smart software add-on, opening up entirely new use cases.

Furthermore, ManoMotion extends its support to stereo RGB cameras and RGBD sensors, enabling even higher precision hand tracking capabilities. We are proud to collaborate with leading players in the professional XR headset market, fostering the integration of the most realistic hand tracking and visualization features in various training and simulation scenarios, including in-cabin experiences and pilot training.