Jonathan Huggins: May 1st

Jonathan Huggins will present his joint work with Frank Wood. Here is an abstract:

We develop a class of non-parametric Bayesian models we call infinite structured explicit duration hidden Markov models (ISEDHMMs). ISEDHMMs are HMMs that possess an unbounded number of states, encode state dwell-time distributions explicitly, and have constraints on what state transitions are allowed. The ISEDHMM framework generalizes explicit duration finite HMMs, infinite HMMs, left-to-right HMMs, and more (all are recoverable by specific choices of ISEDHMM parameters).  This suggests that ISEDHMMs should be applicable to data-analysis problems in a variety of settings.

David Pfau: April 24th

David be presenting "A Spectral Algorithm for Learning Hidden Markov Models" by Hsu, Kakade and Zhang.  The article can be found here.  And the abstract:

Hidden Markov Models (HMMs) are one of the most fundamental and widely used statistical tools for modeling discrete time series. In general, learning HMMs from data is computationally hard (under cryptographic assumptions), and practitioners typically resort to search heuristics which suffer from the usual local optima issues. We prove that under a natural separation condition (bounds on the smallest singular value of the HMM parameters), there is an efficient and provably correct algorithm for learning HMMs. The sample complexity of the algorithm does not explicitly depend on the number of distinct (discrete) observations—it implicitly depends on this quantity through spectral properties of the underlying HMM. This makes the algorithm particularly applicable to settings with a large number of observations, such as those in natural language processing where the space of observation is sometimes the words in a language. The algorithm is also simple: it employs only a singular value decomposition and matrix multiplications.

Yashar Ahmadian: April 10th & 17th

Learning unbelievable marginal probabilities

Xaq Pitkow, Yashar Ahmadian, Ken D. Miller

Loopy belief propagation performs approximate inference on graphical models with loops. One might hope to compensate for the approximation by adjusting model parameters. Learning algorithms for this purpose have been explored previously, and the claim has been made that every set of locally consistent marginals can arise from belief propagation run on a graphical model. On the contrary, here we show that many probability distributions have marginals that cannot be reached by belief propagation using any set of model parameters or any learning algorithm. We call such marginals `unbelievable.' This problem occurs whenever the Hessian of the Bethe free energy is not positive-definite at the target marginals. All learning algorithms for belief propagation necessarily fail in these cases, producing beliefs or sets of beliefs that may even be worse than the pre-learning approximation. We then show that averaging inaccurate beliefs, each obtained from belief propagation using model parameters perturbed about some learned mean values, can achieve the unbelievable marginals.

David Pfau: March 27th (at 5PM)

I'll be presenting on work in progress in collaboration with Bijan Pesaran's group (Yan Wong, Mariana Vigeral, David Putrino) and Josh Merel on building a high degree-of-freedom brain-machine interface.  I'll focus on the Bayesian paradigm for decoding, and two practical problems for pushing that paradigm beyond the commonly-used Kalman filtering approach: building better likelihoods, and building better priors.  The first amounts to fitting tuning curves for various neurons.  Other groups have shown a nonlinear dependence of firing rate on hand position in 3D space, here I will show some preliminary results on fitting tuning curves for large numbers of joint angles.  The second amounts to building better generative models of reach and grasp motions.  As a first step in that direction, I've looked at PCA and ICA for reducing the dimension of reach-and-grasp signals.

Gustavo Lacerda: March 20th

Title: spatial regularization

Consider modeling each neuron as a 2-parameter logistic model (spiking probability as a function of stimulus intensity), and suppose we perform independent experiments on each neuron. Now imagine that the data isn't very informative, so we need to regularize our estimates. We can do spatial regularization by adding a quadratic penalty on the difference of estimates for nearby neurons. Now, suppose that there are *two* types of neurons, and that you only want to shrink together neurons of the same type. We don't want our estimate to be influenced by "false neighbors", i.e. neurons that are spatially close but of a different type. We discuss how to optimize this model. Finally, we explore the idea of Fused Group Lasso.

Kamiar Rahnama Rad: Feb. 21

Two following questions will be discussed:  1. How does embedding low dimensional structures in high dimensional spaces decreases the learning complexity significantly? I will consider the simplest model, that is a linear transformation with additive noise.  2. Modern datasets are accumulated (and in some cases even stored) in a distributed or decentralized manner. Can distributed algorithms be designed to fit a global model over such datasets while retaining the performance of centralized estimators?   

The talk will be based on the following two papers: 

http://www.columbia.edu/~kr2248/papers/ieee-sparse.pdf  

http://www.columbia.edu/~kr2248/papers/CDC2010-1.pdf

Bryan Conroy: Fed. 14th

Bryan Conroy will talk about a fast method for computing many related l2-regularized logistic regression problems, and about possible extensions to other GLMs, and l1-regularizers.