Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications

Esa Ollila; Ammar Mian

doi:10.1109/MLSP49062.2020.9231538

Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications

Esa Ollila, Ammar Mian

Esa Ollila Group

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

2 Citations (Scopus)

80 Downloads (Pure)

Abstract

Huber's criterion can be used for robust joint estimation of regression and scale parameters in the linear model. Huber's [1] motivation for introducing the criterion stemmed from nonconvexity of the joint maximum likelihood objective function as well as non-robustness (unbounded influence function) of the associated ML-estimate of scale. In this paper, we illustrate how the original algorithm proposed by Huber can be set within the block-wise minimization majorization framework. In addition, we propose novel data-adaptive step sizes for both the location and scale, which are further improving the convergence. We then illustrate how Huber's criterion can be used for sparse learning of underdetermined linear model using the iterative hard thresholding approach. We illustrate the usefulness of the algorithms in an image denoising application and simulation studies.

Original language	English
Title of host publication	Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	9781728166629
DOIs	https://doi.org/10.1109/MLSP49062.2020.9231538
Publication status	Published - Sep 2020
MoE publication type	A4 Article in a conference publication
Event	IEEE International Workshop on Machine Learning for Signal Processing - Aalto University, Espoo, Finland Duration: 21 Sep 2020 → 24 Sep 2020 Conference number: 30 https://ieeemlsp.cc

Publication series

Name	IEEE International Workshop on Machine Learning for Signal Processing
ISSN (Print)	2161-0363
ISSN (Electronic)	2161-0371

Workshop

Workshop	IEEE International Workshop on Machine Learning for Signal Processing
Abbreviated title	MLSP
Country/Territory	Finland
City	Espoo
Period	21/09/2020 → 24/09/2020
Internet address	https://ieeemlsp.cc

Keywords

Huber's criterion
Minimization-majorization algorithm
Robust regression
Sparse learning

Access to Document

10.1109/MLSP49062.2020.9231538

Ollila_Block-wise_minimization_IEEEIWoMLSLPAccepted author manuscript, 518 KB

OpenUrl availability

Full text

Cite this

Ollila, E., & Mian, A. (2020). Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications. In Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020 [9231538] (IEEE International Workshop on Machine Learning for Signal Processing). IEEE. https://doi.org/10.1109/MLSP49062.2020.9231538

@inproceedings{ed5768b0b4e043799704ddb858b1a091,

title = "Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications",

abstract = "Huber's criterion can be used for robust joint estimation of regression and scale parameters in the linear model. Huber's [1] motivation for introducing the criterion stemmed from nonconvexity of the joint maximum likelihood objective function as well as non-robustness (unbounded influence function) of the associated ML-estimate of scale. In this paper, we illustrate how the original algorithm proposed by Huber can be set within the block-wise minimization majorization framework. In addition, we propose novel data-adaptive step sizes for both the location and scale, which are further improving the convergence. We then illustrate how Huber's criterion can be used for sparse learning of underdetermined linear model using the iterative hard thresholding approach. We illustrate the usefulness of the algorithms in an image denoising application and simulation studies. ",

keywords = "Huber's criterion, Minimization-majorization algorithm, Robust regression, Sparse learning",

author = "Esa Ollila and Ammar Mian",

year = "2020",

month = sep,

doi = "10.1109/MLSP49062.2020.9231538",

language = "English",

series = "IEEE International Workshop on Machine Learning for Signal Processing",

publisher = "IEEE",

booktitle = "Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020",

address = "United States",

note = "IEEE International Workshop on Machine Learning for Signal Processing, MLSP ; Conference date: 21-09-2020 Through 24-09-2020",

url = "https://ieeemlsp.cc",

}

Ollila, E & Mian, A 2020, Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications. in Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020., 9231538, IEEE International Workshop on Machine Learning for Signal Processing, IEEE, IEEE International Workshop on Machine Learning for Signal Processing, Espoo, Finland, 21/09/2020. https://doi.org/10.1109/MLSP49062.2020.9231538

Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications. / Ollila, Esa; Mian, Ammar.

Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020. IEEE, 2020. 9231538 (IEEE International Workshop on Machine Learning for Signal Processing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications

AU - Ollila, Esa

AU - Mian, Ammar

N1 - Conference code: 30

PY - 2020/9

Y1 - 2020/9

N2 - Huber's criterion can be used for robust joint estimation of regression and scale parameters in the linear model. Huber's [1] motivation for introducing the criterion stemmed from nonconvexity of the joint maximum likelihood objective function as well as non-robustness (unbounded influence function) of the associated ML-estimate of scale. In this paper, we illustrate how the original algorithm proposed by Huber can be set within the block-wise minimization majorization framework. In addition, we propose novel data-adaptive step sizes for both the location and scale, which are further improving the convergence. We then illustrate how Huber's criterion can be used for sparse learning of underdetermined linear model using the iterative hard thresholding approach. We illustrate the usefulness of the algorithms in an image denoising application and simulation studies.

AB - Huber's criterion can be used for robust joint estimation of regression and scale parameters in the linear model. Huber's [1] motivation for introducing the criterion stemmed from nonconvexity of the joint maximum likelihood objective function as well as non-robustness (unbounded influence function) of the associated ML-estimate of scale. In this paper, we illustrate how the original algorithm proposed by Huber can be set within the block-wise minimization majorization framework. In addition, we propose novel data-adaptive step sizes for both the location and scale, which are further improving the convergence. We then illustrate how Huber's criterion can be used for sparse learning of underdetermined linear model using the iterative hard thresholding approach. We illustrate the usefulness of the algorithms in an image denoising application and simulation studies.

KW - Huber's criterion

KW - Minimization-majorization algorithm

KW - Robust regression

KW - Sparse learning

UR - http://www.scopus.com/inward/record.url?scp=85096468569&partnerID=8YFLogxK

U2 - 10.1109/MLSP49062.2020.9231538

DO - 10.1109/MLSP49062.2020.9231538

M3 - Conference contribution

AN - SCOPUS:85096468569

T3 - IEEE International Workshop on Machine Learning for Signal Processing

BT - Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020

PB - IEEE

T2 - IEEE International Workshop on Machine Learning for Signal Processing

Y2 - 21 September 2020 through 24 September 2020

ER -

Block-wise minimization-majorization algorithm for huber's criterion: Sparse learning and applications

Abstract

Publication series

Workshop

Keywords

Access to Document

OpenUrl availability

Other files and links

Fingerprint

Cite this