Authors:	Frank, Alvaro
Title:	Reducing resource waste in HPC through co-allocation, custom checkpoints, and lower false failure prediction rates
Online publication date:	22-Sep-2022
Year of first publication:	2022
Language:	english
Abstract:	Bigger systems are being deployed by High Performance Computing centers in order to fulfill the needs of modern scientific and big data applications as well as to match the increased amount of users in said systems. This thesis explores three methods to reduce wasted computational resources on modern HPC systems with thousands of components. The approaches explored here increase job throughput in HPC systems using co-allocation, reduce unnecessary checkpoints that are triggered after failure predictions and improve checkpoint intervals for common jobs with medium probability of failure. To accomplish these goals, the work first presents a new node sharing strategy for batch systems and shows how it can increase scheduling throughput when compared to standard node allocation methods. Secondly the thesis proposes a new optimal checkpoint interval for jobs with short to medium runtimes that can reduce the expected overhead from checkpointing. Finally it introduces a node failure prediction method tailored to big HPC systems that reduces false positive rates. This thesis offers therefore new insights into the efficiency deficiencies that follow from job failures and resource under-utilization as HPC systems grow in size, while also proposing three techniques that help alleviate said deficiencies.
DDC:	004 Informatik 004 Data processing
Institution:	Johannes Gutenberg-Universität Mainz
Department:	FB 08 Physik, Mathematik u. Informatik
Place:	Mainz
ROR:	https://ror.org/023b0x485
DOI:	http://doi.org/10.25358/openscience-7667
URN:	urn:nbn:de:hebis:77-openscience-5c313850-3828-420a-9f38-f3a45867e2b17
Version:	Original work
Publication type:	Dissertation
License:	CC BY-SA
Information on rights of use:	https://creativecommons.org/licenses/by-sa/4.0/
Extent:	X, 136 Seiten (Illustrationen, Diagramme)
Appears in collections:	JGU-Publikationen