Title: Low-rank Matrix Regularization: Statistical Models and Large Scale Algorithms
Abstract: Low-rank matrix regularization is an important area of research in statistics and machine learning with a wide range of applications --- the task is to estimate X, under a low rank constraint and possibly additional affine (or more general convex) constraints on X. In practice, the matrix dimensions frequently range from hundreds of thousands to even a million --- leading to severe computational challenges. In this talk, I will describe computationally tractable models and scalable (convex) optimization based algorithms for a class of low-rank regularized problems. Exploiting problem-specific statistical insights, problem structure and using novel tools for large scale SVD computations play important roles in this task. I will describe how we can develop a unified, tractable convex optimization framework for general exponential family models, incorporating meta-features on the rows/columns.
Sunday, October 6, 2013
Friday, September 27, 2013
Dean Eckles (Facebook): October 2nd
Title: Design and analysis of experiments in networks
Abstract: Random assignment of individuals to treatments is often used to predict what will happen if the treatment is applied to everyone, but resulting estimates can suffer substantial bias in the presence of peer effects (i.e., interference, spillovers, social interactions). We describe experimental designs that reduce this bias by producing treatment assignments that are correlated in the network. For example, we can use graph partitioning methods to construct clusters of individuals who are then assigned to treatment or control together. This clustered assignment alone can substantially reduce bias, as can incorporating information about peers' treatment assignments or behaviors into the analysis. Simulation results show how this bias reduction varies with network structure and the size of direct and peer effects. We illustrate this method with real experiments, including a large experiment on Thanksgiving Day 2012.
Abstract: Random assignment of individuals to treatments is often used to predict what will happen if the treatment is applied to everyone, but resulting estimates can suffer substantial bias in the presence of peer effects (i.e., interference, spillovers, social interactions). We describe experimental designs that reduce this bias by producing treatment assignments that are correlated in the network. For example, we can use graph partitioning methods to construct clusters of individuals who are then assigned to treatment or control together. This clustered assignment alone can substantially reduce bias, as can incorporating information about peers' treatment assignments or behaviors into the analysis. Simulation results show how this bias reduction varies with network structure and the size of direct and peer effects. We illustrate this method with real experiments, including a large experiment on Thanksgiving Day 2012.
Sunday, September 22, 2013
Donald Pianto: September 25th
Title: Dealing with monotone likelihood in a model for speckled data
Abstract: In this paper we study maximum likelihood estimation (MLE) of the roughness parameter of the G_{A}^{0} distribution for speckled imagery (Frery et al., 1997). We discover that when a certain criterion is satisfied by the sample moments, the likelihood function is monotone and MLE estimates are infinite, implying an extremely homogeneous region. We implement three corrected estimators in an attempt to obtain finite parameter estimates. Two of the estimators are taken from the literature on monotone likelihood (Firth, 1993; Jeffreys, 1946) and one, based on resampling, is proposed by the authors. We perform Monte Carlo experiments to compare the three estimators. We find the estimator based on the Jeffreys prior to be the worst. The choice between Firth’s estimator and the Bootstrap
estimator depends on the value of the number of looks (which is given before estimation) and the specific needs of the user. We also apply the estimators to real data obtained from synthetic aperture radar (SAR). These results corroborate the Monte Carlo findings.
Abstract: In this paper we study maximum likelihood estimation (MLE) of the roughness parameter of the G_{A}^{0} distribution for speckled imagery (Frery et al., 1997). We discover that when a certain criterion is satisfied by the sample moments, the likelihood function is monotone and MLE estimates are infinite, implying an extremely homogeneous region. We implement three corrected estimators in an attempt to obtain finite parameter estimates. Two of the estimators are taken from the literature on monotone likelihood (Firth, 1993; Jeffreys, 1946) and one, based on resampling, is proposed by the authors. We perform Monte Carlo experiments to compare the three estimators. We find the estimator based on the Jeffreys prior to be the worst. The choice between Firth’s estimator and the Bootstrap
estimator depends on the value of the number of looks (which is given before estimation) and the specific needs of the user. We also apply the estimators to real data obtained from synthetic aperture radar (SAR). These results corroborate the Monte Carlo findings.
Sunday, September 15, 2013
Prof. John Paisley: September 18th
Title: Variational Inference and Big Data
Abstract: I will discuss a scalable algorithm for approximating posterior distributions called stochastic variational inference. Stochastic variational inference lets one apply complex Bayesian models to massive data sets. This technique applies to a large class of probabilistic models and outperforms traditional batch variational inference, which can only handle small data sets. Stochastic inference is a simple modification to the batch approach, so a significant part of the discussion will focus on reviewing this traditional batch inference method.
Abstract: I will discuss a scalable algorithm for approximating posterior distributions called stochastic variational inference. Stochastic variational inference lets one apply complex Bayesian models to massive data sets. This technique applies to a large class of probabilistic models and outperforms traditional batch variational inference, which can only handle small data sets. Stochastic inference is a simple modification to the batch approach, so a significant part of the discussion will focus on reviewing this traditional batch inference method.
Friday, September 6, 2013
David Carlson: September 11th
Title: Real-Time Inference for a Gamma Process Model of Neural Spiking
Abstract: With simultaneous measurements from ever increasing populations of neurons, there is a growing need for sophisticated tools to recover signals from individual neurons. In electrophysiology experiments, this classically proceeds in a two-step process: (i) threshold the waveforms to detect putative spikes and (ii) cluster the waveforms into single units (neurons). We extend previous Bayesian nonparametric models of neural spiking to jointly detect and cluster neurons using a Gamma process model. We develop an online approximate inference scheme enabling real-time analysis, with performance exceeding the previous state-of-the-art. Via exploratory data analysis we find several features of our model collectively contribute to our improved performance including: (i) accounting for colored noise, (ii) detecting overlapping spikes, (iii) tracking waveform dynamics, and (iv) using multiple channels.
In my talk, I will give a brief overview of the Bayesian nonparametric structures that have been used in the spike-sorting problem. From there, I will give details on how we've taken the spike sorting model and integrated it with a Poisson process to improve the noisy detection problem, and give details on learning the model using real-time online methods. Additionally, I will discuss extensions to evolving waveform dynamics and multiple channels, and present results from a tetrode as well as from novel 3-channel and 8-channel multi-electrode arrays where action potentials may appear on some but not all of the channels.
Abstract: With simultaneous measurements from ever increasing populations of neurons, there is a growing need for sophisticated tools to recover signals from individual neurons. In electrophysiology experiments, this classically proceeds in a two-step process: (i) threshold the waveforms to detect putative spikes and (ii) cluster the waveforms into single units (neurons). We extend previous Bayesian nonparametric models of neural spiking to jointly detect and cluster neurons using a Gamma process model. We develop an online approximate inference scheme enabling real-time analysis, with performance exceeding the previous state-of-the-art. Via exploratory data analysis we find several features of our model collectively contribute to our improved performance including: (i) accounting for colored noise, (ii) detecting overlapping spikes, (iii) tracking waveform dynamics, and (iv) using multiple channels.
In my talk, I will give a brief overview of the Bayesian nonparametric structures that have been used in the spike-sorting problem. From there, I will give details on how we've taken the spike sorting model and integrated it with a Poisson process to improve the noisy detection problem, and give details on learning the model using real-time online methods. Additionally, I will discuss extensions to evolving waveform dynamics and multiple channels, and present results from a tetrode as well as from novel 3-channel and 8-channel multi-electrode arrays where action potentials may appear on some but not all of the channels.
Tuesday, July 23, 2013
Prof. Qi Wang (Biomedical Engineering, Columbia): July 24th
Title: Reading and Writing the Neural Code: Initial Steps toward Engineered Sensory Percepts
Abstract: The transformation of sensory signals into spatiotemporal patterns of neural activity in the brain is critical in forming our perception of the external world. Physical signals, such as light, sound, and force, are transduced to neural electrical impulses, or spikes, at the periphery, and these spikes are subsequently transmitted to the brain through various stages of the sensory pathways, ultimately forming the representation of the sensory world. Deciphering the information conveyed in the spike trains is often referred to as “reading the neural code”. On the other hand, prosthetic devices designed to restore lost sensory function, such as cochlear implants, rely primarily on the principle of artificially activating neural circuits to induce a desired perception, which we might refer to as “writing the neural code”. This requires not only significant challenges in biomaterials and interfaces, but also in knowing precisely what to tell the brain to do.
My talk will focus on three topics. First, I will talk about the control of peripheral tactile sensations. Specifically, I will discuss the synthesis of virtual tactile sensations using a custom-built, high spatiotemporal resolution tactile display, a device we designed to create high fidelity, computer-controlled tactile sensations on the fingertip similar to those arising naturally. Second, I will utilize a decoding paradigm to discuss the neural representations of tactile sensations and how they are encoded and transformed across early stages of processing in the somatosensory pathway. Finally, I will discuss the design of sub-cortical microstimulation to control cortical activation, using downstream cortical measurements as a benchmark of the fidelity of the surrogate signaling. Taken together, an understanding of how to read and write the neural code is essential not only for the development of technologies for translating thoughts into actions (motor prostheses), but also for the development of technologies for creating artificial sensory percepts (sensory prostheses).
Abstract: The transformation of sensory signals into spatiotemporal patterns of neural activity in the brain is critical in forming our perception of the external world. Physical signals, such as light, sound, and force, are transduced to neural electrical impulses, or spikes, at the periphery, and these spikes are subsequently transmitted to the brain through various stages of the sensory pathways, ultimately forming the representation of the sensory world. Deciphering the information conveyed in the spike trains is often referred to as “reading the neural code”. On the other hand, prosthetic devices designed to restore lost sensory function, such as cochlear implants, rely primarily on the principle of artificially activating neural circuits to induce a desired perception, which we might refer to as “writing the neural code”. This requires not only significant challenges in biomaterials and interfaces, but also in knowing precisely what to tell the brain to do.
My talk will focus on three topics. First, I will talk about the control of peripheral tactile sensations. Specifically, I will discuss the synthesis of virtual tactile sensations using a custom-built, high spatiotemporal resolution tactile display, a device we designed to create high fidelity, computer-controlled tactile sensations on the fingertip similar to those arising naturally. Second, I will utilize a decoding paradigm to discuss the neural representations of tactile sensations and how they are encoded and transformed across early stages of processing in the somatosensory pathway. Finally, I will discuss the design of sub-cortical microstimulation to control cortical activation, using downstream cortical measurements as a benchmark of the fidelity of the surrogate signaling. Taken together, an understanding of how to read and write the neural code is essential not only for the development of technologies for translating thoughts into actions (motor prostheses), but also for the development of technologies for creating artificial sensory percepts (sensory prostheses).
Tuesday, July 9, 2013
Carl Smith: July 9th
Title: Low-rank graphical models and Bayesian inference in the statistical analysis of noisy neural data
Abstract: We develop new methods of Bayesian inference, largely in the context of analysis of neuroscience data. The work is broken into several parts. In the first part, we introduce a novel class of joint probability distributions in which exact inference is tractable. Previously it has been difficult to find general constructions for models in which efficient exact inference is possible, outside of certain classical cases. We identify a class of such models that are tractable owing to a certain “low-rank” structure in the potentials that couple neighboring variables. In the second part we develop methods to quantify and measure information loss in analysis of neuronal spike train data due to two types of noise, making use of the ideas developed in the first part. Information about neuronal identity or temporal resolution may be lost during spike detection and sorting, or precision of spike times may be corrupted by various effects. We quantify the information lost due to these effects for the relatively simple but sufficiently broad class of Markovian model neurons. We find that decoders that model the probability distribution of spike-neuron assignments significantly outperform decoders that use only the most likely spike assignments. We also apply the ideas of the low-rank models from the first section to defining a class of prior distributions over the space of stimuli (or other covariate) which, by conjugacy, preserve the tractability of inference. In the third part, we treat Bayesian methods for the estimation of sparse signals, with application to the locating of synapses in a dendritic tree. We develop a compartmentalized model of the dendritic tree. Building on previous work that applied and generalized ideas of least angle regression to obtain a fast Bayesian solution to the resulting estimation problem, we describe two other approaches to the same problem, one employing a horseshoe prior and the other using various spike-and-slab priors. In the last part, we revisit the low-rank models of the first section and apply them to the problem of inferring orientation selectivity maps from noisy observations of orientation preference. The relevant low-rank model exploits the self-conjugacy of the von Mises distribution on the circle. Because the orientation map model is loopy, we cannot do exact inference on the low-rank model by the forward back- ward algorithm, but block-wise Gibbs sampling by the forward backward algorithm speeds mixing. We explore another von Mises coupling potential Gibbs sampler that proves to effectively smooth noisily observed orientation maps.
Abstract: We develop new methods of Bayesian inference, largely in the context of analysis of neuroscience data. The work is broken into several parts. In the first part, we introduce a novel class of joint probability distributions in which exact inference is tractable. Previously it has been difficult to find general constructions for models in which efficient exact inference is possible, outside of certain classical cases. We identify a class of such models that are tractable owing to a certain “low-rank” structure in the potentials that couple neighboring variables. In the second part we develop methods to quantify and measure information loss in analysis of neuronal spike train data due to two types of noise, making use of the ideas developed in the first part. Information about neuronal identity or temporal resolution may be lost during spike detection and sorting, or precision of spike times may be corrupted by various effects. We quantify the information lost due to these effects for the relatively simple but sufficiently broad class of Markovian model neurons. We find that decoders that model the probability distribution of spike-neuron assignments significantly outperform decoders that use only the most likely spike assignments. We also apply the ideas of the low-rank models from the first section to defining a class of prior distributions over the space of stimuli (or other covariate) which, by conjugacy, preserve the tractability of inference. In the third part, we treat Bayesian methods for the estimation of sparse signals, with application to the locating of synapses in a dendritic tree. We develop a compartmentalized model of the dendritic tree. Building on previous work that applied and generalized ideas of least angle regression to obtain a fast Bayesian solution to the resulting estimation problem, we describe two other approaches to the same problem, one employing a horseshoe prior and the other using various spike-and-slab priors. In the last part, we revisit the low-rank models of the first section and apply them to the problem of inferring orientation selectivity maps from noisy observations of orientation preference. The relevant low-rank model exploits the self-conjugacy of the von Mises distribution on the circle. Because the orientation map model is loopy, we cannot do exact inference on the low-rank model by the forward back- ward algorithm, but block-wise Gibbs sampling by the forward backward algorithm speeds mixing. We explore another von Mises coupling potential Gibbs sampler that proves to effectively smooth noisily observed orientation maps.
Subscribe to:
Posts (Atom)