Regularization, Optimization, Kernels, and Support Vector Machines

Regularization, Optimization, Kernels, and Support Vector Machines offers a snapshot of the current state of the art of large-scale machine learning, providing a single multidisciplinary source for the latest research and advances in regularization, sparsity, compressed sensing, convex and large-scale optimization, kernel methods, and support vector machines. Consisting of 21 chapters authored by leading researchers in machine learning, this comprehensive reference: Covers the relationship between support vector machines (SVMs) and the Lasso Discusses multi-layer SVMs Explores nonparametric feature selection, basis pursuit methods, and robust compressive sensing Describes graph-based regularization methods for single- and multi-task learning Considers regularized methods for dictionary learning and portfolio selection Addresses non-negative matrix factorization Examines low-rank matrix and tensor-based models Presents advanced kernel methods for batch and online machine learning, system identification, domain adaptation, and image processing Tackles large-scale algorithms including conditional gradient methods, (non-convex) proximal techniques, and stochastic gradient descent Regularization, Optimization, Kernels, and Support Vector Machines is ideal for researchers in machine learning, pattern recognition, data mining, signal processing, statistical learning, and related areas.

Contents Preface Contributors An Equivalence between the Lasso and Support Vector Machines; Martin Jaggi Regularized Dictionary Learning; Annalisa Barla, Saverio Salzo, and Alessandro Verri Hybrid Conditional Gradient-Smoothing Algorithms with Applications to Sparse and Low Rank Regularization; Andreas Argyriou, Marco Signoretto, and Johan A.K. Suykens Nonconvex Proximal Splitting with Computational Errors; Suvrit Sra Learning Constrained Task Similarities in Graph-Regularized Multi-Task Learning; Rémi Flamary, Alain Rakotomamonjy, and Gilles Gasso The Graph-Guided Group Lasso for Genome-Wide Association Studies; Zi Wang and Giovanni Montana On the Convergence Rate of Stochastic Gradient Descent for Strongly Convex Functions; Cheng Tang and Claire Monteleoni Detecting Ineffective Features for Nonparametric Regression; Kris De Brabanter, Paola Gloria Ferrario, and László Györfi Quadratic Basis Pursuit; Henrik Ohlsson, Allen Y. Yang, Roy Dong, Michel Verhaegen, and S. Shankar Sastry Robust Compressive Sensing; Esa Ollila, Hyon-Jung Kim, and Visa Koivunen Regularized Robust Portfolio Estimation; Theodoros Evgeniou, Massimiliano Pontil, Diomidis Spinellis, Rafal Swiderski, and Nick Nassuphis The Why and How of Nonnegative Matrix Factorization; Nicolas Gillis Rank Constrained Optimization Problems in Computer Vision; Ivan Markovsky Low-Rank Tensor Denoising and Recovery via Convex Optimization; Ryota Tomioka, Taiji Suzuki, Kohei Hayashi, and Hisashi Kashima Learning Sets and Subspaces; Alessandro Rudi, Guillermo D. Canas, Ernesto De Vito, and Lorenzo Rosasco Output Kernel Learning Methods; Francesco Dinuzzo, Cheng Soon Ong, and Kenji Fukumizu Kernel Based Identification of Systems with Multiple Outputs Using Nuclear Norm Regularization; Tillmann Falck, Bart De Moor, and Johan A.K. Suykens Kernel Methods for Image Denoising; Pantelis Bouboulis and Sergios Theodoridis Single-Source Domain Adaptation with Target and Conditional Shift; Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, Zhikun Wang, Zhi-Hua Zhou, and Claudio Persello Multi-Layer Support Vector Machines; Marco A. Wiering and Lambert R.B. Schomaker Online Regression with Kernels; Steven Van Vaerenbergh and Ignacio Santamaría Index

Johan A.K. Suykens is a professor at Katholieke Universiteit Leuven, Belgium, where he obtained a degree in electro-mechanical engineering and a Ph.D in applied sciences. He has been a visiting postdoctoral researcher at the University of California, Berkeley, USA, and a postdoctoral researcher with the Fonds Wetenschappelijk Onderzoek - Vlaanderen, Belgium. A senior IEEE member, he has co/authored and edited several books; received many prestigious awards; directed, co/organized, and co/chaired numerous international conferences; and served as associate editor for the IEEE Transactions on Circuits and Systems and the IEEE Transactions on Neural Networks. Marco Signoretto is currently a visiting lecturer at the Centre for Computational Statistics and Machine Learning (CSML), University College London, UK, in the framework of a postdoctoral fellowship with the Belgian Fund for Scientific Research (FWO). He holds a Ph.D in mathematical engineering from Katholieke Universiteit Leuven, Belgium; a degree in electronic engineering (Laurea Magistralis) from the University of Padova, Italy; and an M.Sc in methods for management of complex systems from the University of Pavia, Italy. His research interests include practical and theoretical aspects of mathematical modeling of structured data, with special focus on multivariate time-series, networks, and dynamical systems. His current work deals with methods based on (convex) optimization, structure-inducing penalties, and spectral regularization. Andreas Argyriou has received degrees in computer science from the Massachusetts Institute of Technology, Cambridge, USA, and a Ph.D in computer science from University College London (UCL), UK. The topic of his Ph.D work has been on machine learning methodologies integrating multiple tasks and data sources. He has held postdoctoral and research faculty positions at UCL; Toyota Technological Institute at Chicago, Illinois, USA; and Katholieke Universiteit Leuven, Belgium. He is currently serving an RBUCE-UP fellowship at École Centrale Paris, France. His current interests are in the areas of kernel methods, multitask learning, compressed sensing, and convex optimization methods.