COVID-19, the IT development perspective and the role of 3D hand tracking

16/04/2020

It is predictable that humans’ daily routines and how we will be living after Corona time will be significantly affected by the fear of a pandemic. Digital tools and infrastructure such as video conferencing solutions and high-speed Internet connections can facilitate the way people deal with the current situation. However, radical changes and enforced shifts in technology require longer adaptation time from people until the next generation fully adapts to the new culture. It is expected that specific areas of IT and smart technologies go through an extremely faster development due to the urgent demands to help similar situations after the COVID-19. Here is our shortlist of some of those areas.  

Winners of Corona and Post-Corona time

With no doubt, the pharmaceutical industry and medical sector are the frontiers of the industries that make a direct impact in controlling and handling the epidemic situation. However, today it is hard to believe that any industry, including the medical sector, can survive without some aspect of smart technologies, artificial intelligence, and data-driven solutions. Automated data collection, data mining, and data-driven statistical analysis are some of the latest advancements that help governments and scientists identify different aspects and monitor the epidemic situation intelligently. In addition to the medical sector that will be highly driven by the developments of new technologies we foresee the following application domains as the next important areas to be followed and the top candidates to receive significant attention and investment.   

Online events, meetings and the role of XR

Nowadays, one of the most effective ways to control the COVID-19 spread is by working remotely. In short term analysis, video communication software solutions such as Skype and Zoom are the winners of the situation. Organizing millions of online events per day is possible with these great video communication technologies. However, looking at the problem in the long-term, the lack of feeling of realism and presence in the existing online communication tools is a huge drawback. XR technologies are superb for this use case, in order to provide this important feature, by enhancing the feeling of presence in mixed reality. Imagine wearing a smart glass or headset and see your colleagues’ holograms around you in a shared space, being able to call them and re-create a similar experience as the real-world physical presence. Although this idea sounds futuristic, it is certainly the next step for all scientists and tech giants in the XR industry. Creating VR meetings and avatars might be the transition to the holographic and mixed reality events.      

Education and E-learning

As one of the early adopters of technology, schools, universities and research centers are already experiencing a totally new way of organizing daily work including giving lectures, seminars, taking exams and one to one online communications. Obviously, with the developments of the network technologies remote work has been experienced for a long time but enforcing a large part of the society to shift their daily work online is certainly a new challenge. Since this shift has shown great success, many people question the regular education systems on how this has not happened earlier in order to save a huge amount of cost and effort on a daily commute, premises, administration, etc. The key factor in many educational and research fields is the need for physical interaction for labs, workshops, and training sessions. The medical field is also a big exception as the contact between students and patients is the key part of the education. The next big move in e-learning will be related to re-creating the workshops and educational practices in virtual or augmented space. Collaborative AR/VR experience is certainly an important feature of e-learning. The presence of a user’s hand and natural interaction with hands are the key factors of success.   

The long-term fear of touching in public places, touchscreen displays and physical interaction

A critical aspect to control the spread of Coronavirus is to avoid or minimize the touch of surfaces in populated areas and public places. The aftermath of the epidemic situation will be the concern for the fear of touching and spreading of the virus and will remain in the human mind. But what is the effective move to minimize the touch in public places? The answer is applying the touchless technology to a wide range of areas that require physical touch today. Touchscreen public displays in shopping centers and restaurants, all payments devices with card and pin code, all physical buttons in metros, elevators and many more physical interactions can be shifted to a touchless version. Part of the physical interaction can be shifted to personalized devices and mobile applications instead of using public devices, i.e. mobile payments instead of card payments and mobile apps for transportation instead of charging and using cards. Part of the physical interaction can be performed with hand tracking i.e. camera-based menu selection in restaurants with bare hands in 3D space or in-air selection of buttons in an elevator by hand tracking technology. Obviously other sensor technologies such as proximity sensors can handle such cases as well.     

Virtual try-ons and online shopping

A huge part of the shopping, especially clothes, are already shifted to online stores. With the Corona situation, this shift has been dramatically faster and we expect that in the near future major part of the cloth shopping will happen online. However, trying clothes virtually is a major challenge due to the limitations of the technology to augment the items on the body in real-time with high precision and realistic visualization.    

Today, virtual try-ons are one of the most interesting applications of mobile AR. The existing applications use live camera feed to augment an object on body parts. Examples include wearing an augmented shoe, nail polish, face masks, and eyeglasses. There are clear requests for technologies to support the virtual hand wearables such as virtual gloves, watches, and rings. 3D hand tracking in AR is a must-have feature to support hand wearables. Obviously to provide a proper user experience tracking finger joints and hand orientation is quite important for designers to be able to augment virtual objects on top of the hand.    

How ManoMotion technology can boost this dramatic technology shift? 

Considering the above points, remote presence and touch-less interaction are a clear path for development in smart technologies. Hand tracking is an essential tool for both cases as visualization, presence, and interaction with hands are a necessity in any XR environment. ManoMotion is on the mission to provide real-time full hand tracking and gesture analysis on affordable smart devices. We support major mobile and AR/VR platforms. For remote presence, real-time hand and skeleton visualizations are the main required features. For touch-less interaction, tracking hands and understanding the dynamic gestures such as pick, drop, select, grab are required.

ManoMotion offers two standard solutions for mobile platforms to support the above-mentioned scenarios. With our latest premium product, SDK PRO, we extend our existing technologies of hand tracking and gesture analysis. We now support full skeleton tracking and additional details about joint positions & orientations.

Shahrouz Yousefi, CTO, ManoMotion AB

Tags: