Thermal camera imaging - Autonomous and Cyber-Physical Systems (ACPS) Research Group

The thesis titled “Hand Gestures Recognition using Thermal Images” done by the master student, Daniel Skomedal Breland under the supervision Prof. Linga Reddy Cenkeramaddi has been awarded the best master thesis in ICT for the year 2021.

Hand Gestures Recognition using Thermal Imaging.

The goal of this project is to develop a robust and reliable hand gesture recognition system using a thermal camera. Hand gestures are an important communication tool for many practical scenarios. It is used in a variety of applications, including medical, entertainment, and industrial settings. The use of human-robot interactions is growing, and there exists several methods. It is possible to gain access to tight and harsh places by using gestures. The majority of gesture recognition is done with RGB cameras, which has the disadvantage of not being able to recognize gestures in low-light situations. Thermal cameras can operate in low-light environments because they are unaffected by external light.

Daniel S. Breland, Simen B. Skriubakken, Aveen Dayal, Ajit Jha, Phaneendra K. Yalavarthy, and Linga R. Cenkeramaddi, “Deep Learning based Sign Language Digits Recognition from Thermal Images with Edge Computing System,” IEEE Sensors Journal 2021 (in press).

Keywords: Gesture recognition, Cameras, Pins, Integrated circuit modeling, Assistive technology, Three-dimensional displays, Lighting

Abstract: The sign language digits based on hand gestures have been utilized in various applications such as human-computer interaction, robotics, health and medical systems, health assistive technologies, automotive user interfaces, crisis management and disaster relief, entertainment, and contactless communication in smart devices. The color and depth cameras are commonly deployed for hand gesture recognition, but the robust classification of hand gestures under varying illumination is still a challenging task. This work presents the design and deployment of a complete end-to-end edge computing system that can accurately provide the classification of hand gestures captured from thermal images. A thermal dataset of 3200 images was created with each sign language digit having 320 thermal images. The solution presented here utilizes live images taken from a low-resolution thermal camera of 32×32 pixels, feeding into a novel light weight deep learning model based on bottleneck motivated from deep residual learning for classification of hand gestures. The edge computing system presented here utilizes Raspberry pi with a thermal camera making it highly portable. The designed system achieves an accuracy of 99.52% on the test data set with an added advantage of accuracy being invariable to background lighting conditions as it is based on thermal imaging.

More details:DOI: 10.1109/JSEN.2021.3061608

Tag: Thermal camera imaging

The Best Master’s Thesis in Information and Communication Technologies – 2021

The article, “Deep Learning based Sign Language Digits Recognition from Thermal Images with Edge Computing System,” has been accepted for publication in the IEEE Sensors Journal.