TY - CHAP

T1 - Why LASSO, EN, and CLOT

T2 - Invariance-Based Explanation

AU - Alkhatib, Hamza

AU - Neumann, Ingo

AU - Kreinovich, Vladik

AU - Van Le, Chon

N1 - Funding Information: Acknowledgements This work was supported by the Institute of Geodesy, Leibniz University of Hannover. It was also supported in part by the US National Science Foundation grants 1623190 (A Model of Change for Preparing a New Generation for Professional Practice in Computer Science) and HRD-1242122 (Cyber-ShARE Center of Excellence). This paper was written when V. Kreinovich was visiting Leibniz University of Hannover.

PY - 2020/11/14

Y1 - 2020/11/14

N2 - In many practical situations, observations and measurement results are consistent with many different models—i.e., the corresponding problem is ill-posed. In such situations, a reasonable idea is to take into account that the values of the corresponding parameters should not be too large; this idea is known as regularization. Several different regularization techniques have been proposed; empirically the most successful are LASSO method, when we bound the sum of absolute values of the parameters, and EN and CLOT methods in which this sum is combined with the sum of the squares. In this paper, we explain the empirical success of these methods by showing that they are the only ones which are invariant with respect to natural transformations—like scaling which corresponds to selecting a different measuring unit.

AB - In many practical situations, observations and measurement results are consistent with many different models—i.e., the corresponding problem is ill-posed. In such situations, a reasonable idea is to take into account that the values of the corresponding parameters should not be too large; this idea is known as regularization. Several different regularization techniques have been proposed; empirically the most successful are LASSO method, when we bound the sum of absolute values of the parameters, and EN and CLOT methods in which this sum is combined with the sum of the squares. In this paper, we explain the empirical success of these methods by showing that they are the only ones which are invariant with respect to natural transformations—like scaling which corresponds to selecting a different measuring unit.

UR - http://www.scopus.com/inward/record.url?scp=85096207492&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-48853-6_2

DO - 10.1007/978-3-030-48853-6_2

M3 - Contribution to book/anthology

AN - SCOPUS:85096207492

SN - 978-3-030-48852-9

T3 - Studies in Computational Intelligence

SP - 37

EP - 50

BT - Data Science for Financial Econometrics

PB - Springer Science and Business Media Deutschland GmbH

CY - Cham

ER -