ThermoNet: deep neural network thermogram analysis of human calves during physical exercise
Date issued
Authors
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
License
Abstract
Applied infrared thermography allows practitioners and researchers to evaluate human thermoregulation and gain physiological insights based on pattern recognition of non-invasive acquired thermograms. Current research in sports science and medicine already utilizes thermography in several areas, including injury detection and prevention, disease detection and monitoring, as well as understanding the metabolism and physiology of individuals under external physical load or stress. Studies are limited by manual image selection and analysis or the application of specialized hand-crafted algorithms to detect regions of interest. Thermal features are extracted and analyzed from a few thermogram samples. This dissertation proposes the end-to-end acquisition and segmentation pipeline "ThermoNet" to acquire radiometrically calibrated thermograms, segment regions of interest, automatically extract thermal features, and fuse them with additional external sensor data such as heart rate or breath analysis. An entire experiment, measured with a high-speed, high-resolution thermographic camera, is now fully analyzable, instead of being examined only on cherry-picked samples. Contrary to common practice, radiometric calibration is performed in each thermogram using a custom two-point calibration device. Regions of interest include body part extraction, i.e. left and right calf, and vascular-related patterns: superficial vein and perforator patterns. The patterns are additionally analyzed among their individual instances, allowing for further differentiation in explaining thermoregulatory processes.
Two specialized deep neural networks semantically segment the thermograms. Therefore, this thesis explores the development of these networks, including the construction of appropriate manually annotated datasets. The work focuses on the backside of runners on a treadmill to evaluate their calves. Other regions of interest are not yet included. To mitigate the lack of initial datasets for these regions, a method for bootstrapping an annotated dataset based on a stereo system with a thermal camera and a visual + depth camera is presented. Application of the system results in automatically annotated datasets that provide a starting point for new segmentation models and reduce the need for large manually annotated datasets.
The processing pipeline "ThermoNet" allows analysts to apply further investigation to the time series of an entire experiment. Several studies revealed relationships between skin temperature radiation and other physiological attributes. Thus, the work integrates into several areas of sports science and medicine.