Dissertation defense (August 21, 2024): Luis Carlos Ramos Alvarenga

Student: Luis Carlos Ramos Alvarenga

Title: Resource Dimensioning for Execution of Scientific Workflows in High-Performance Computing Environments

Advisors: Rafaelli de Carvalho Coutinho and Daniel Cardoso de Moraes de Oliveira

Committee: Rafaelli de Carvalho Coutinho (PPCIC), Daniel Cardoso de Moraes de Oliveira (UFF), Marta Lima de Queirós Mattoso (UFRJ), Pedro Henrique González Silva (PPCIC), Yuri Abitbol de Menezes Frota (UFF)

Day/Time: August 21, 2024 / 9 a.m.

Room: https://teams.microsoft.com/l/meetup-join/19%253aafLsm4pQ8LA4BQvp0ARudoFA5U9d26I1HIyxCW0qO5w1%2540thread.tacv2/1721171058293?context%3D%257b%2522Tid%2522%253a%25228eeca404-a47d-4555-a2d4-0f3619041c9c%2522%252c%2522Oid%2522%253a%252254af42a0-5f30-4905-ac8d-10b96c6db26b%2522%257d&sa=D&source=calendar&usd=2&usg=AOvVaw0kLstrc7AyYeq7jI–b_7E

Abstract: Scientists increasingly need to run highly computationally demanding experiments. These experiments, often modeled as scientific workflows, are run in high-performance computing (HPC) environments. Typically, these environments provide a wide range of resources to users. The proper sizing of resources for running scientific workflows in these environments is a crucial task. An undersized or oversized environment can directly affect the performance of the experiment, leading to negative impacts on the time and financial cost of execution. Thus, research involving resource estimation for experiment execution in HPC environments has been conducted, such as the GraspCC heuristic, which uses the adaptive randomized greedy search procedure (GRASP).
The objective of this work is to investigate the execution of scientific workflows in high-performance environments, such as clusters and computer clouds, in order to efficiently estimate the required resources considering the associated time and financial costs. The problem was defined from the proposed structuring of workflow into parallel stages or levels of tasks that are similar to each other and operate independently, called Layered-Bucket (LB).
To solve the problem, we proposed an integer mathematical programming formulation and an adaptation of the GraspCC heuristic to accommodate the LB approach, named GraspCC-LB. The proposed approach was evaluated using real traces of workflows from the fields of bioinformatics and astronomy. The resource estimations produced by GraspCC-LB were compared against the actual resource usage in a real-world HPC environment to assess its effectiveness. The results demonstrate the effectiveness of GraspCC-LB as a robust approach for resource optimization in the context of large-scale scientific workflows that require HPC capabilities, serving as an important decision-support tool.