1 . A research team led by Professors Ki-Hun Jeong and Doheon Lee from the Korea Advanced Institute of Science and Technology (KAIST) Department of Bio and Brain Engineering reported the development of a technique for facial expression detection by mixing light-field camera techniques with artificial intelligence (AI) technology.

Unlike a conventional camera, the light-field camera contains sets of micro-lens (微透镜) in front of the image sensor, which makes the camera small enough to fit into a smart phone while allowing it to acquire the spatial and directional information of the light with a single shot. The technique has received attention as it can reconstruct images in a variety of ways including multi-views, refocusing and 3D image acquisition, giving rise to many potential applications.

However, the optical (光学的) crosstalk between shadows caused by external light sources in the environment and the micro-lens has limited existing light-field cameras from being able to provide accurate image contrast and 3D reconstruction.

The research team applied a laser in the near-infrared (NIR) range to stabilize the accuracy of 3D image reconstruction that previously depended on environmental light. When an external light source is shone on a face at 0-30, and 60-degree angles, the light-field camera reduces 54% of the image reconstruction errors. Additionally, by inserting a light-absorbing layer for visible and NIR wavelengths between the sets of micro-lens, the team could minimize optical crosstalk while increasing the image contrast by 2.1 times.

Through this technique, the team could overcome the limitations of existing light-field cameras and develop a more advanced NIR-based light-field camera (NIR-LFC) improved for the 3D image reconstruction of facial expressions. Using the NIR-LFC, the team acquired high-quality 3D reconstruction images of facial expressions expressing various emotions regardless of the lighting conditions of the surrounding environment.

The facial expressions in the acquired 3D images were distinguished through machine learning with an average of 85% accuracy-a statistically significant figure compared to when 2D images were used. Furthermore, by calculating the inter-dependency of distance information that varies with facial expression in 3D images, the team could identify the information alight-field camera uses to distinguish human expressions.