Modeling MST Neuron Receptive Fields



Abstract (AIPRW 2009):

Biologically inspired approaches are an alternative to conventional engineering approaches when developing complex algorithms for intelligent systems. In this paper, we present a novel approach to the computational modeling of primate cortical neurons in the dorsal medial superior temporal area (MSTd). Our approach is based-on a spatially distributed mixture of Gaussians, where MST's primary function is detecting self-motion from optic flow stimulus. Each biological neuron was modeled using a genetic algorithm to determine the parameters of the mixture of Gaussians, resulting in firing rate responses that accurately match the observed responses of the corresponding biological neurons. We also present the possibility of applying the trained models to machine vision as part of a simple dorsal stream processing model for self-motion detection, which as applications to motion analysis and unmanned vehicle navigation.


MST Neuronal Receptive Field:

Dual-Gaussian Model: We model each MST neuron's receptive field using a 3x3 grid, each containing a two-component Gaussian mixture. The Gaussians can be either positive or negative, that corresponds to excitatory and inhibitory effects of a region. The models are trained according to the pre-recorded neuronal firing rates on monkeys, and optimized using the genetic algorithm. Figure 1 shows the dual Gaussians of each RF segment, each Gaussian is designed to have a wrap-around" effect, with the x-axis being the input motion stimulus (-180 to 180 degrees) and the y-axis the firing rate.


Figure 1. A sample MST neuron RF model: final firing rate is obtained by summing over the effects from all 9 segments. Note that in addition to the planar addative effects, we also modeled the segmental-interaction effects in our 2010 work.

Training and Predicting: Each RF's segments are trained separately using the pre-recorded neuronal firing rate data, with motion stimulus cued at inidivdual segments; then the entire 3x3 trained RF is used to predict the "flow" firing rates that had the stimulus occupying the entire viewing angle. Figure 2 shows that the additive effect obtained by summing over all segments, that the whole-field motion stimulus firing rate of the same neuron was well predicted by our model.


Figure 2. Left: the single-segment training result; Middle: the whole-field firing rate prediction result; Right: corresponding RF model.

The Dorsal-Stream Model: Dorsal-stream is the visual pathway that is also known as the "where" pathway, its main responsibility is to process the visual motion perceived by the human eyes. We use our MST neuron RF models that were trained using real recorded firing rate data as part of this novel dorsal-stream model, and apply it to classify different self-motion stimulus (forward, turning left, turning right, backward) presented from videos (Figure 3).


Figure 3. Our dorsal-stream model for detecting self-motions. Each video is pre-processed into corner keypoints, that's filtered with the Gaussian-derivative filters [1][2]; the results are used as motion stimuli for the trained MST neuronal RF models. The outputed euronal firing rates are used as inputs to a neural network for the final self-motion classification (4 outputs: turning left, turning right, backward, forward).


Sample Results and Videos (Click on the images for video):

   videos require Xvid MPEG-4 codec, can be downloaded from

   Dorsal-stream Model Testing Result (tested on a virtual reality of Strong Memorial Hospital sequence)



   [1] R. A. Young. The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spatial Vision, 1987.

   [2] R. A. Young and R. M. Lesperance. The gaussian derivative model for spatial-temporal vision: II. Cortical data.
   Spatial Vision, 2001.