Kolia Sadeghi : August 4

again

I will attempt to take the latest paper on the deterministic particle flow filter discussed in a previous blog post, and strip it down to the essentials.  The authors present a more general, stable and improved version of their previous deterministic particle flow filter, supposedly. This paper is rife with ideas and peculiarly written; for a gentler introduction, please refer to the papers linked to in the previous blog post. Here is the paper:

Exact particle flow for nonlinear filters by Fred Daum, Jim Huang and Arjang Noushin,
Numerical experiments for nonlinear filters with exact particle flow induced by log-homotopy (companion paper)

A Computationally Efficient Method for Nonparametric Modeling of Neural Spiking Activity with Point Processes

this would be interesting to go over at some point...

A Computationally Efficient Method for Nonparametric Modeling of
Neural Spiking Activity with Point Processes
Todd Coleman and Sridevi Sarma

Deterministic particle filtering

No resampling, rejection, or importance sampling are used. Particles are propagated through time by numerically integrating an ODE. The method is very similar in spirit to Jascha Sohl-Dickstein, Peter Battaglino and Mike DeWeese's Minimum probability flow learning, but applied to nonlinear filtering.


The authors report orders of magnitude speedups for higher dimensional state spaces where sampling rejection would be a problem.

Particle flow for nonlinear filters with log-homotopy by Fred Daum & Jim Huang

There are a couple of papers companion to this one:
Nonlinear filters with particle flow induced by log-homotopy

Seventeen dubious methods to approximate the gradient for nonlinear filters with particle flow


As you may see, the authors have a very peculiar writing style.


However, one very recent paper by Lingji Chen and Raman Mehra points out some flaws in the approach:
A study of nonlinear filters with particle flow induced by log-homotopy
(but see the group meeting announcement above for Fred Daum and Jim Huang's recent answer to this).

Kolia Sadeghi : July 28

I will present work done with Liam, Jeff Gauthier and others in EJ Chichilnisky's lab on locating retinal cones from multiple ganglion cell recordings.  We write down a single hierarchical model where ganglion cell responses are modeled as independent GLMs with space-time-color separable filters and no spike history.  Assuming the stimulus was gaussian ensures that the ganglion cell Spike Triggered Averages are sufficient statistics.  The spatial component is then assumed to be a weighted sum of non-overlapping and appropriately placed archetypical cone receptive fields.  With a benign approximation, we can integrate out the weights and focus on doing MCMC in the space of cone locations and colors only.  As it turns out, this likelihood landscape has many nasty local maxima; we use parallel tempering and a few techniques specific to this problem to ensure ergodicity of the markov chain.

Doing a google scholar search on parallel tempering, also known as replica exchange, or just exchange Monte Carlo, will bring up many papers on this simple technique. Here is a review:
Parallel tempering: Theory, applications, and new perspectives

Some classic stats papers

These are a bit more old-school, but still of interest:

http://www.ceremade.dauphine.fr/~xian/M2classics

Some interesting papers from AISTATS 2010

Here are a few potentially interesting papers from AISTATS this year. All pdf's available from

http://jmlr.csail.mit.edu/proceedings/papers/v9/ - in no particular order:

Improving posterior marginal approximations in latent Gaussian models

by Botond Cseke, Tom Heskes

Dirichlet Process Mixtures of Generalized Linear Models

by Lauren Hannah, David Blei, Warren Powell

Simple Exponential Family PCA

by Jun Li, Dacheng Tao

Convex Structure Learning in Log-Linear Models: Beyond Pairwise Potentials

by Mark Schmidt, Kevin Murphy

Reduced-Rank Hidden Markov Models

by Sajid Siddiqi, Byron Boots, Geoffrey Gordon

Detecting Weak but Hierarchically-Structured Patterns in Networks

by Aarti Singh, Robert Nowak, Robert Calderbank

Inference of Sparse Networks with Unobserved Variables. Application to Gene Regulatory Networks

by Nikolai Slavov

On the relation between universality, characteristic kernels and RKHS embedding of measures

by Bharath Sriperumbudur, Kenji Fukumizu, Gert Lanckriet

State-Space Inference and Learning with Gaussian Processes

by Ryan Turner, Marc Deisenroth, Carl Rasmussen

Parallelizable Sampling of Markov Random Fields

by James Martens, Ilya Sutskever

Approximation of hidden Markov models by mixtures of experts with application to particle filtering

by Jimmy Olsson, Jonas Strojby

Negative Results for Active Learning with Convex Losses

by Steve Hanneke, Liu Yang

Jittering spike trains carefully

Presented in lab meeting by Alex Ramirez on July 14th, 2010.

A Rate and History-Preserving Resampling Algorithm for Neural Spike Trains

by Matthew Harrison and Stuart Geman

In it the authors describe an algorithm that takes a spike train and jitters the spike times to create a new spike train which is maximally random while preserving the firing rate and recent spike-history of the original train.