David Pfau: Dec. 20th

David will be giving a fly-by view of a number of cool papers from NIPS.

First is Empirical Models of Spiking in Neural Populations by Macke, Büsing, Cunningham, Yu, Shenoy and Mahani, where they evaluate the relative merits of GLMs with pairwise coupling and state space models on multielectrode recording in motor cortex.

Next, Quasi-Newton Methods for Markov Chain Monte Carlo by Zhang and Sutton looks at how to use approximate second-order methods like L-BFGS for MCMC while still preserving detailed balance.

Then, Demixed Principal Component Analysis is an extension of PCA which demixes the dependence of different latent dimensions on different observed parameters, and is used to analyze neural data from PFC

Finally, Learning to Learn with Compound Hierarchical-Deep Models, which combines a deep neural network for learning visual features with a hierarchical nonparametric Bayesian model for learning object categories to make one cool-looking demo.

Alex Ramirez: June 21th

Alex will be presenting a short version of this paper. In it the authors consider loss functions, for many estimators, that obey certain smoothness and convexity requirements and prove a global, geometric convergence (fast) rate of convergence under Nestorv's Gradient descent method up to a level of Statistical precision.

There will be no meeting on June 14th.

Max Nikitchenko: May 10

This Tuesday, on 2011/05/10, I will discuss methods for the acceleration of the convergence of algorithms with linear convergence near the fixed point, such as EM, which are known to be notoriously slow in that area. Two approaches are possible: modify the iterative algorithm itself (PX-EM (by parameter-expansion), ECM (by maximizing the maximizer individually for each parameter, keeping the others fixed), etc), or use the recent history of the iterations to extrapolate them closer to the fixed point (in which case you keep all your machinery intact and only plug in an auxiliary function for extrapolating the already computed iteration steps). I will talk about the second class of the accelerators.

I will start with the method I derived myself, which is visual, but powerful at the same time. I will then focus on two papers which seem to become the gold standard in the acceleration techniques: Varadhan, R. & Roland, C. "Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm" (dx.doi.org/10.1111/j.1467-9469.2007.00585.x) from 2008 and Zhou, H.; Alexander, D. & Lange, K. "A quasi-Newton acceleration for high-dimensional optimization algorithms" (dx.doi.org/10.1007/s11222-009-9166-3) from 2011. I have just found out about the second paper and it seems to overlap heavily with the method I derived. I hope we will clear this question up!

Jianing Shi : April 26th

I will discuss Nesterov's optimal gradient method at the group meeting.  I will talk about Nesterov's method for minimizing composite objective function, together with its implication for L1 minimization. 

There is unfortunately no short story on Nesterov's method, however you can find his work at

http://www.core.ucl.ac.be/~nesterov/

Jianing's nicely done slides can be found here.

Jonathan Huggins: April 5

This week I will be talking about submodular set functions, which possess a useful and intuitive diminishing returns property. I will begin with the definition and give a variety of examples of situations in which submodular functions arise. I'll discuss some connections to convex and concave functions, as well as strategies for minimization and maximization. I will mainly draw from the classic paper "Submodular functions and convexity" by Lovász, as well as a recent paper by Stobbe and Krause, which provides a nice summary of key results, as well as a discussion of many of the advances since the Lovász paper. If the talk sparks your interest, then I highly recommend the tutorial (complete with hours of video!) by Krause and Guestrin.

Eftychios Pnevmatikakis: Sept. 8

I will present the recent paper on computationally efficient methods for nonparametric modeling of neural spiking activity with point processes by Coleman and Sarma. A copy can be found here.