Communications in Information and Systems
Volume 18 (2018)
An investigation for loss functions widely used in machine learning
Pages: 37 – 52
Over past few decades, numerous machine learning algorithms have been developed for solving various problems arising in practical applications. And, loss function is one of the most significant factors influencing the performance of algorithm. Nevertheless, most readers may be confused about the reason why these loss functions are effective in corresponding models. The confusion further interfere them to select reasonable loss functions for their algorithms. In this paper, we take a comprehensive investigation for some representative loss functions and analyse the latent properties of them. One of the goals of the investigation is to find the reason why bilateral loss functions are more suitable for regression task, while unilateral loss functions are more suitable for classification task. In addition, a significant question we discuss is that how to judge the robustness of a loss function. The investigation is useful for readers to develop or improve their future works.
This work was supported in part by the National Natural Science Foundation of China grant under numbers 61772427 and 61751202.