Title: Regularized Learning in Reproducing Kernel Banach Spaces
Speaker: Jun Zhang
Department of Psychology, University of Michigan, Ann Arbor
Time: 11 am, Wednesday, Aug. 22
Venue: Room 1801, East Guanghua Tower, Handan Campus, Fudan University
Abstract:
Regularized learning is the contemporary framework for learning to generalize from finite samples (classification, regression, clustering, etc). Here the problem is to learn an inputoutputmappingf: X>Y, either scalarvalued or vectorvalued, given finite samples {(xi, yi), i=1,…,N}. With minimal structural assumptions on X, the class of functions under consideration is assumed to fall under a Banach (especially, Hilbert) space of functions B. The learningfromdata problem is then formulated as an optimization problem in such a function space, with the desired mapping as an optimizer to be sought, where the objective function consists of a loss term L(f) capturing itsgoodnessoffit (or the lack thereof) on given samples {(f(xi), yi), i=1,…,N}, and a penalty term R(f) capturing its complexity based on prior knowledge about the solution (smoothness,sparsity, etc). This second, regularizing term is often taken to be the norm of B, or an innocent transformation thereof: R(f) = (f). This program has been successfully carried out for the Hilbert space of functions, resulting in the celebrated Reproducing Kernel Hilbert Space methods in machine learning. Here, we will remove the Hilbert space restriction, i.e., the existence of an inner product, and show that the key ingredients of this framework (reproducing kernel, representer theorem, feature space) remain to hold for a Banach space that is uniformly convex and uniformly Frechet differentiable. Central to our development is the use of a semiinner product operator and duality mapping for a uniform Banach space in place of an innerproduct for a Hilbert space. This opens up the possibility of unifying kernelbased methods (regularizing L2norm) and sparsitybased methods (regularizing l1norm), which have so far been investigated under different theoretical foundations.
