Ari Pakman: Oct. 10th

Ari will present the paper "The Horseshoe Estimator for Sparse Signals"

Link: http://faculty.chicagobooth.edu/nicholas.polson/research/papers/Horse.pdf

Abstract:

This paper proposes a new approach to sparse-signal detection called the horseshoe estimator. We show that the horseshoe is a close cousin of the lasso in that it arises from the same class of multivariate scale mixtures of normals, but that it is almost universally superior to the double-exponential prior at handling sparsity. A theoretical framework is proposed for understanding why the horseshoe is a better default “sparsity” estimator than those that arise from powered-exponential priors. Comprehensive numerical evidence is presented to show that the difference in performance can often be large. Most importantly, we show that the horseshoe estimator corresponds quite closely to the answers one would get if one pursued a full Bayesian model-averaging approach using a “two-groups” model: a point mass at zero for noise, and a continuous density for signals. Surprisingly, this correspondence holds both for the estimator itself and for the classification rule induced by a simple threshold applied to the estimator. We show how the resulting thresholded horseshoe can also be viewed as a novel Bayes multiple-testing procedure.

Gustavo Lacerda: March 20th

Title: spatial regularization

Consider modeling each neuron as a 2-parameter logistic model (spiking probability as a function of stimulus intensity), and suppose we perform independent experiments on each neuron. Now imagine that the data isn't very informative, so we need to regularize our estimates. We can do spatial regularization by adding a quadratic penalty on the difference of estimates for nearby neurons. Now, suppose that there are *two* types of neurons, and that you only want to shrink together neurons of the same type. We don't want our estimate to be influenced by "false neighbors", i.e. neurons that are spatially close but of a different type. We discuss how to optimize this model. Finally, we explore the idea of Fused Group Lasso.

Kolia Sadeghi : Sept. 20

This week, I'll be giving a fly-by overview of a string of recent papers on exact sparse signal recovery that do better than LASSO by solving a sequence of L1 or L2 penalized problems.  Here is a basic narrative:

LASSO uses a penalty weighted by the same lambda for all coefficients.  What happens if you assign different lambdas to each coefficient, and update these lambdas iteratively?  Candes and Boyd do this in Enhancing sparsity by reweighted L1 minimization

You can obtain sparsity by iterative reweighting even for L2-penalized problems: if some of the lambdas become infinite, the corresponding coefficients become exactly zero.  Chartrand and Yin find a particularly good L2 reweighing scheme in Iteratively reweighted algorithms for compressive sensing

All of the above methods reweigh each lambda based only on the value of its corresponding coefficient: they are separable.  In Iterative reweighted l1 and l2 methods for finding sparse solutions, Wipf considers non-separable reweighting schemes that come out of Sparse Bayesian Learning (SBL), which you might also know by the name of Relevance Vector Machine or Automatic Revelance Determination.