David Pfau : Nov 17th

Abstract:

I'm presenting joint work with Frank Wood and Nicholas Bartlett on learning simple models for discrete sequence prediction.  We describe a novel Bayesian framework for learning probabilistic deterministic finite automata (PDFA), which are a class of simple generative models for sequences from a discrete alphabet.  We first define a prior over PDFA with a fixed number of states, and then by taking the limit as the number of states becomes unbounded, we show that the prior has a well defined limit, a model we call a Probabilistic Deterministic Infinite Automata (PDIA).  Inference is tractable with MCMC, and we show results from experiments with synthetic grammars, DNA and natural language.  In particular, we find on complex data that averaging predictions over many MCMC samples leads to improved performance, and that the learned models perform as well as 3rd-order Markov models with about 1/10th as many states.  For the curious, a write-up of my work can be found here.

Also, following the talk I'm going to give a brief tutorial on git, a free version control system used in the software community for maintaining large collaborative code bases.  I'd like to set up a git repository for the Paninski group so we can avoid too much code duplication and build on each others' work, and I promise it's actually pretty easy once you learn the basics.

Scalable inference on regularized Dirichlets

Hi all,

I'm giving a brief (30") talk about my recent work this Wednesday at noon in room 903 SSW. The abstract is below. I'll presumably be giving a fuller talk on the same eventually in our group meeting, but in case you're looking for lunchtime entertainment...

Cheers,
Carl

P.S. This is a one-hour situation with two half-hour presenters. So if you come, you could wind up watching someone else first, or only!


Title: Tractable inference on regularized Dirichlet distributions: a scalable class of HMM

Abstract: There is substantial interest in tractable inference on distributions of distributions, confined obviously to a simplex. Regularization of the Dirichlet distribution of random variables, without compromising tractability of inference, would be useful for encoding prior knowledge of interactions among the components, for instance in topic models. I will present a class of regularized Dirichlet distributions that are in fact especially scalable hidden Markov models. The same framework allows for tractable exact inference on certain loopy graphs of the same type.

Kamiar Rahnama Rad and Chaitu Ekanadham: Nov 10

Chaitu will describe the major results and proofs since the 2005 paper by Candes which gives sufficient conditions for stable recovery of sparse signals from incomplete measurements.  Kamiar will be finishing off where he left off at his last presentation.

Paper by Machens may be of interest...

Demixing population activity in higher cortical areas

Christian Machens

Micky Vidne: October 27th. s(MC)^2 or Hesitant Particle Filter.

In my talk I will describe a recent extension of the Sequential Monte Carlo (SMC) method. SMCs (particle filters) are a commonly used method to estimate a latent dynamical process from sequential noise-contaminated observations. SMCs are extremely powerful but suffer from sample impoverishment, a situation in which very few different particles represent the distribution of interest. I will describe our attempt to circumvent this fundamental problem by adding an extra MCMC step in the SMC algorithm. I will illustrate the usefulness of this algorithm by considering a toy neuroscience example.

Kolia Sadeghi : Oct 20th

I'll be going over this paper on Deep Boltzmann Machines and adaptive MCMC starting with some background on Restricted Boltzmann Machines.  In passing I'll give a quick overview of related architectures used to learn temporal sequences.

Kamiar Rahnama Rad : 4pm Oct 13

Kamiar will be going over the following paper at 4pm:

M. J. Wainwright (2009), Sharp thresholds for noisy and high-dimensional recovery of sparsity using L1-constrained quadratic programming (Lasso). , IEEE Transactions on Information Theory, 55:2183--2202, May 2009