Hyperparameter Optimization: Enhancing Model Efficiency and Accuracy in Machine Learning

Context

Hyperparameter Optimization (HPO) is essential for optimizing machine learning models by tuning parameters that significantly impact their performance. Unlike trainable parameters, hyperparameters are set before training, influencing the model behavior and performance. Advancements in HPO algorithms aim to automate and improve model performance across different tasks.
In this thesis, you will explore various HPO algorithms, including traditional methods like grid search and random search, as well as advanced techniques such as Bayesian Optimization and meta-learning approaches. The goal is to investigate how HPO techniques enhance the performance and efficiency of machine learning models by tuning critical hyperparameters such as learning rate, batch size and network architecture, identifying strengths and weaknesses in terms of efficiency, scalability, and effectiveness in achieving optimal model performance. Additionally, you can evaluate these HPO techniques using frameworks like Optuna, Hyperopt etc. Visualization for both process and results can also be part of the thesis.

Tasks

• Literature research of the SOTA approaches 
• Implementation and comparison of HPO algorithms using machine learning frameworks
• Design of experiments to evaluate the impact of different hyperparameters on model training and performance, with process and result visualization
• Evaluation and analysis of experimental results to obtain insights into the effectiveness of HPO methods in improving model outcomes

Requirements

• High motivation to learn new technologies
• Knowledge of machine learning algorithms 
• Experience with programming languages such as Python, C++ and Java
• Analytical, problem-solving, and communication skills