Berlin Lectures on Neurotechnology
Location
-
Lecture Hall of the
BCCN Berlin
- Bernstein Center for Computational Neuroscience
- Humboldt University
- Philippstr. 13 House 6
- 10115 Berlin
Language:
English
Dates
- November 23rd, 2011 at 11 AM (s.t.)
-
Learning from small samples in high dimensions
-
Lars Kai Hansen
, DTU Informatics, Technical University of Denmark
- I will discuss recent progress in coping with variance inflation in
high-dimensional unsupervised learning (PCA and kPCA).
Small sample high-dimensional principal component analysis (PCA)
suffers from variance inflation and lack of generalizability. It has
earlier been pointed out that a simple leave-one-out variance
renormalization scheme can cure the problem. We have generalized the
cure in two directions: First, we propose a computationally less
intensive approximate leave-one-out estimator, secondly, we show that
variance inflation is also present in kernel principal component
analysis (kPCA) and we provide a non-parametric renormalization scheme
which can quite efficiently restore generalizability in kPCA. As for
PCA our analysis also suggests a simplified approximate
expression. Finally, I present evidence that these ideas may be
relevant also for supervised learning in SVM's.
T.J. Abrahamsen and L.K. Hansen.
A Cure for Variance Inflation in High Dimensional Kernel Principal
Component Analysis. Journal of Machine Learning Research 12:2027-2044
(2011).
