Kamiar Rahnama Rad: Feb. 21

Two following questions will be discussed:  1. How does embedding low dimensional structures in high dimensional spaces decreases the learning complexity significantly? I will consider the simplest model, that is a linear transformation with additive noise.  2. Modern datasets are accumulated (and in some cases even stored) in a distributed or decentralized manner. Can distributed algorithms be designed to fit a global model over such datasets while retaining the performance of centralized estimators?   

The talk will be based on the following two papers: 

http://www.columbia.edu/~kr2248/papers/ieee-sparse.pdf  

http://www.columbia.edu/~kr2248/papers/CDC2010-1.pdf

Bryan Conroy: Fed. 14th

Bryan Conroy will talk about a fast method for computing many related l2-regularized logistic regression problems, and about possible extensions to other GLMs, and l1-regularizers.

Eftychios P.: Jan. 31st and Feb. 7th

I am planning to lead a very informal discussion on some neat techniques for convex and semidefinite relaxation that can be used to transform intractable optimization problems into approximate but convex ones. I'll also discuss a few applications to statistical neuroscience that we are currently pursuing.

Some background material (although I'm not planning to go over any of these in detail) includes:

http://www.se.cuhk.edu.hk/~manchoso/papers/sdrapp-SPM.pdf
http://arxiv.org/abs/1012.0621
http://www-stat.stanford.edu/~candes/papers/PhaseRetrieval.pdf
http://users.cms.caltech.edu/~jtropp/papers/MT11-Two-Proposals-EJS.pdf

David Pfau: Dec. 20th

David will be giving a fly-by view of a number of cool papers from NIPS.

First is Empirical Models of Spiking in Neural Populations by Macke, Büsing, Cunningham, Yu, Shenoy and Mahani, where they evaluate the relative merits of GLMs with pairwise coupling and state space models on multielectrode recording in motor cortex.

Next, Quasi-Newton Methods for Markov Chain Monte Carlo by Zhang and Sutton looks at how to use approximate second-order methods like L-BFGS for MCMC while still preserving detailed balance.

Then, Demixed Principal Component Analysis is an extension of PCA which demixes the dependence of different latent dimensions on different observed parameters, and is used to analyze neural data from PFC

Finally, Learning to Learn with Compound Hierarchical-Deep Models, which combines a deep neural network for learning visual features with a hierarchical nonparametric Bayesian model for learning object categories to make one cool-looking demo.

Previous Group Meetings (for archival purposes)

Universal MAP Estimation in Compressed Sensing, by Baron and Duarte

Quantifying Statistical Interdependence by Message Passing on Graphs, by Dauwels, Vialatte, Weber and Chichocki. Part I and Part II

The No-U_Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo, by Hoffman and Gelman.

Ari Pakman: Dec. 13th

"Rescaling, thinning or complementing? On goodness-of-fit procedures for point process models and Generalized Linear Models" by Gerhard and Gerstner (NIPS 2010).

The link is http://books.nips.cc/papers/files/nips23/NIPS2010_0767.pdf

The abstract reads:

"Generalized Linear Models (GLMs) are an increasingly popular framework for modeling neural spike trains. They have been linked to the theory of stochastic point processes and researchers have used this relation to assess goodness-of-fit using methods from point-process theory, e.g. the time-rescaling theorem. However, high neural firing rates or coarse discretization lead to a breakdown of the assumptions necessary for this connection. Here, we show how goodness-of-fit tests from point-process theory can still be applied to GLMs by constructing equivalent surrogate point processes out of time-series observations. Furthermore, two additional tests based on thinning and complementing point processes are introduced. They augment the instruments available for checking model adequacy of point processes as well as discretized models."

Kolia Sadeghi : Sept. 20

This week, I'll be giving a fly-by overview of a string of recent papers on exact sparse signal recovery that do better than LASSO by solving a sequence of L1 or L2 penalized problems.  Here is a basic narrative:

LASSO uses a penalty weighted by the same lambda for all coefficients.  What happens if you assign different lambdas to each coefficient, and update these lambdas iteratively?  Candes and Boyd do this in Enhancing sparsity by reweighted L1 minimization

You can obtain sparsity by iterative reweighting even for L2-penalized problems: if some of the lambdas become infinite, the corresponding coefficients become exactly zero.  Chartrand and Yin find a particularly good L2 reweighing scheme in Iteratively reweighted algorithms for compressive sensing

All of the above methods reweigh each lambda based only on the value of its corresponding coefficient: they are separable.  In Iterative reweighted l1 and l2 methods for finding sparse solutions, Wipf considers non-separable reweighting schemes that come out of Sparse Bayesian Learning (SBL), which you might also know by the name of Relevance Vector Machine or Automatic Revelance Determination.