Coordinated multiplexing of information about separate objects in visual cortex - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov 29:11:e76452.
doi: 10.7554/eLife.76452.

Coordinated multiplexing of information about separate objects in visual cortex

Na Young Jun  1   2   3 , Douglas A Ruff  4   5 , Lily E Kramer  4   5 , Brittany Bowes  4   5 , Surya T Tokdar  6 , Marlene R Cohen  4   5 , Jennifer M Groh  1   2   3   7   8   9
Affiliations

Coordinated multiplexing of information about separate objects in visual cortex

Na Young Jun et al. Elife. .

Abstract

Sensory receptive fields are large enough that they can contain more than one perceptible stimulus. How, then, can the brain encode information about each of the stimuli that may be present at a given moment? We recently showed that when more than one stimulus is present, single neurons can fluctuate between coding one vs. the other(s) across some time period, suggesting a form of neural multiplexing of different stimuli (Caruso et al., 2018). Here, we investigate (a) whether such coding fluctuations occur in early visual cortical areas; (b) how coding fluctuations are coordinated across the neural population; and (c) how coordinated coding fluctuations depend on the parsing of stimuli into separate vs. fused objects. We found coding fluctuations do occur in macaque V1 but only when the two stimuli form separate objects. Such separate objects evoked a novel pattern of V1 spike count ('noise') correlations involving distinct distributions of positive and negative values. This bimodal correlation pattern was most pronounced among pairs of neurons showing the strongest evidence for coding fluctuations or multiplexing. Whether a given pair of neurons exhibited positive or negative correlations depended on whether the two neurons both responded better to the same object or had different object preferences. Distinct distributions of spike count correlations based on stimulus preferences were also seen in V4 for separate objects but not when two stimuli fused to form one object. These findings suggest multiple objects evoke different response dynamics than those evoked by single stimuli, lending support to the multiplexing hypothesis and suggesting a means by which information about multiple objects can be preserved despite the apparent coarseness of sensory coding.

Keywords: multiplexing; neuroscience; noise correlations; object vision; population coding; rhesus macaque; variability.

PubMed Disclaimer

Conflict of interest statement

NJ, DR, LK, BB, ST, MC No competing interests declared, JG Reviewing editor, eLife

Figures

Figure 1.
Figure 1.. Experimental design.
(a) Multiunit activity was recorded in V1 and V4 using chronically implanted 10 × 10 or 6 × 8 electrode arrays in six monkeys (see ‘Methods’). (b) In both ‘superimposed’ and ‘adjacent’ datasets, the stimuli were positioned to overlap with (‘adjacent’ dataset) or completely span (‘superimposed’ dataset) the centers of the receptive fields of the recorded neurons. (c) In the ‘superimposed’ dataset, gratings were presented either individually or in combination at a consistent location and were large enough to cover the V1 and V4 receptive fields (stimulus diameter range: 2.5–7o). The combined gratings appeared as a plaid (rightmost panel). Monkeys maintained fixation throughout stimulus presentation and performed no other task. (d) In the V1 ‘adjacent’ dataset, Gabor patches were smaller (typically ~1o, see Figure 1—figure supplement 1) and were presented individually or side-by-side roughly covering the region of the V1 receptive fields. Monkeys maintained fixation while performing an orientation change detection task. The data analyzed in this study involved trials in which the monkeys were attending a third Gabor patch located in the ipsilateral hemifield to perform the orientation change detection. (e) In the V4 ‘adjacent’ dataset, the stimuli consisted of either Gabor patches or natural image stimuli, and monkeys performed a fixation task. Incorrectly performed trials and stimulus presentations during which we detected microsaccades were excluded from all analyses.
Figure 1—figure supplement 1.
Figure 1—figure supplement 1.. Stimulus and receptive field positions for the V1 "adjacent stimuli" dataset.
(a) Example receptive field positions, sample estimated receptive field size, and layout of stimuli for the V1 dataset involving adjacent stimuli. This figure was adapted from Figure 1B of Ruff and Cohen, 2016. (b) Actual stimuli used. One ‘A’ and one ‘B’ stimulus was used in each session. The sizes of the circles indicate the sizes of the Gabor patches; specifically, the radii are equal to twice the standard deviations of the Gaussian envelopes used to construct the patches. Some stimuli were centered slightly into the ipsilateral hemifield but extended into the contralateral hemifield.
Figure 1—figure supplement 2.
Figure 1—figure supplement 2.. Eye positions did not differ on single vs. combined stimulus trials (adjacent dataset).
(a) Average eye positions during stimulus presentations on single (gray) vs. dual (magenta) stimulus trials during one recording session. Box indicates the fixation window, which was ±0.5°. (b) Geometric mean of the horizontal and vertical standard deviations of eye position for this session. (c) Average standard deviation of eye position across all 16 recording sessions as a function of stimulus type. The single vs. combined stimulus values did not differ (one-tailed paired t-test, p=0.927).
Figure 1—figure supplement 3.
Figure 1—figure supplement 3.. Relationship between receptive field (RF) location, stimulus location, spike count model classification , and whether firing rates also correlated with scatter in eye position.
For clarity, receptive field centers are shown in relationship to the location of the more contralateral of the two stimuli; the other stimulus location is shown as the average of all the other stimulus locations in relation to the most contralateral one. Each dot shows the RF center of one unit condition (i.e., unit and particular set of stimulus conditions tested in the analysis). Yellow-orange dots correspond to the RF centers for unit conditions that met criteria for inclusion but for which the Bayesian response pattern classification did not yield a high-confidence result (i.e., probability of successful classification was less than 0.67). Magenta and green symbols indicate RF centers for unit conditions that were classified as ‘mixture’ (magenta) or as something other than a ‘mixture’ (green) with high confidence (probability of successful classification was greater than 0.67). The magenta and green bull’s eye symbols correspond to unit conditions that also showed a significant correlation between firing rate and fixational scatter along the dimension connecting the two stimuli for that session (p<0.01). A small number of unit conditions are not plotted here if the RF mapping did not produce a good estimate of RF center. Overall, significant correlations between spike count and scatter of eye fixation (p<0.01) were identified in 4% of the stimulus conditions that were included for analysis (57/1389, see Table 1). Among the conditions that could be successfully categorized with a probability of at least 0.67 (see Figure 2), the prevalence of significant eye position correlations did not differ significantly for conditions labels as ‘mixtures’ (~9%) vs. those that received some other label (~5%). These proportions did not differ from one another by chi-square test (p=0.1730), nor is there any clear pattern in the location of unit conditions showing correlations with eye scatter in relation to RF centers and stimulus locations as shown in this figure. Finally, about 9% of ‘mixtures’ (n = 9) were responsive to both stimuli (i.e., RF centers were intermediate between the two stimuli and responses exceeded baseline firing by at least 1 standard deviation for both stimuli alone); among these nine, only one showed sensitivity to eye position (11%).
Figure 2.
Figure 2.. Examples of V1 units showing fluctuating activity pattern and formal statistical analysis.
(a) Distribution of spike counts on single stimuli (red, blue) and dual adjacent stimulus presentations (black) for three units in V1 tested with adjacent stimuli. Spikes were counted in a 200 ms window following stimulus onset. (b) Bayesian model comparison regarding spike count distributions. We evaluated the distribution of spike counts on combined stimulus presentations in relation to the distributions observed on when individual stimuli were presented alone. Four possible models were considered as described in the equations and text. Only one case each of the ‘single’ (B-like) and ‘outside’ (λAB > max(λA, λB) is shown. (c, d) Best spike count models for the adjacent (c) and superimposed (d) stimulus datasets, meeting a minimum winning probability of at least 0.67, i.e., the winning model is at least twice as likely as the best alternative. Pie chart insets illustrate proportion of tested conditions that met this confidence threshold. While ‘singles’ dominated in the adjacent stimulus dataset and ‘singles’ and ‘outsides’ dominated in the superimposed stimulus datasets, we focus on the presence of a ‘mixtures’ as an important minority subpopulation present nearly exclusively in the ‘adjacent’ stimulus dataset.
Figure 2—figure supplement 1.
Figure 2—figure supplement 1.. Detailed results of the spike count response pattern classification analysis on V1 units for the adjacent (a) and superimposed datasets (b).
Shading indicates the confidence level of the model categorization. A winning probability of >0.5 indicates that the winning model is at least as likely as all other models combined; a probability of >0.67 indicates the winner is at least twice as likely as all others, and a probability of 0.95 indicates it is at least 20 times as likely as all others. Cases with a winning probability of >0.25 (the minimum possible) but less than 0.5 are not shown. As noted in the main text, for the adjacent dataset, ‘mixtures’ represented 33% of the cases in which a particular model was at least twice as likely as all other models combined (i.e., the winning probability of 0.67); overall they represented 14% of the cases that passed the exclusion criteria as described in ‘Methods’ and Table 1.
Figure 3.
Figure 3.. Schematic depiction of possible response patterns and resulting correlations.
(a) Three hypothetical neurons and their possible spike count distributions for single-stimulus presentations. Units 1 and 2 both respond better to stimulus ‘A’ than to stimulus ‘B’ (“congruent” preferences). Unit 3 shows the opposite pattern (‘incongruent’ preferences). (b, c) Possible pairwise spike count correlation (Rsc) patterns for these units. Two units that have congruent A vs. B response preferences will show positive correlations with each other if they both show ‘A-like’ or ‘B-like’ activity on the same trials (panel b, left). In contrast, if one unit prefers ‘A’ and the other ‘B’ (incongruent), then A-like or B-like activity in both units on the same trial will produce a negative spike count correlation (panel b, right). The opposite pattern applies when units tend to respond to different stimuli on different trials (panel c). (d–f). Key examples of the inferences to be drawn at the population level from these potential correlation patterns. (d) Positive correlations among ‘congruent’ pairs negative correlations among ‘incongruent’ pairs would suggest only one stimulus is encoded at the population level at a time. (e) If both stimuli are encoded in the population, then both positive and negative correlations might be observed among both congruent and incongruent pairs. (f) Both stimuli may be encoded, but not necessarily equally. This example shows a pattern intermediate between the illustrations in (d) and (e), and is consistent with one of the two stimuli being overrepresented compared to the other. Other possibilities exist as well, including that neurons may could be uncorrelated with one another (not shown), which would also serve to preserve information about both stimuli at the population level.
Figure 3—figure supplement 1.
Figure 3—figure supplement 1.. Details of population-level predictions under different scenarios.
Schematic histograms of the possible patterns of spike count correlations across all the pairs of recorded neurons as a function of whether they share the same (‘congruent’) stimulus preferences or have different (‘incongruent’) preferences, and whether they tend to ‘encode’ the same stimulus or different stimuli on each presentation (panel h is panel a + panel d; i = a + e; j = b + f; k = c + g; the bottom row [c, g, k] is a biased version of the row above [b, f, j]). Panels (h), (j), and (k) are included in Figure 3 as panels (d), (e), and (f).
Figure 4.
Figure 4.. Patterns of spike count correlations among pairs of V1 neurons in different subgroups and conditions.
(a) Example units’ correlation patterns (same units as Figure 2a). The two units that shared a similar tuning preference (‘congruent’) exhibited positively correlated spike count variation on individual stimulus presentations for the dual stimulus condition (left), whereas both units 1 and 2 exhibited a negative correlation with the differently tuned (‘incongruent’) unit 3 (middle and right panels). (b) Distribution of Rsc values for the other units that were simultaneously recorded with unit 1 and were also classified as ‘mixtures,’ color coded according to whether the stimulus preference of the other unit was the same as that of unit 1 (‘congruent,’ green) or different (‘incongruent,’ brown). All of the ‘congruent’ pairs exhibited positive correlations, and 5 of 5 were individually significant (p<0.05). All of the ‘incongruent’ pairs exhibited negative correlations, and 1 of 5 was individually significant (p<0.05). (c) Overall, neural pairs in which both units met the ‘mixture’ classification showed distinct positive and negative patterns of correlation in response to adjacent stimuli. Positive correlations were more likely to occur among pairs of neurons that responded more strongly to the same individual stimuli (‘congruent,’ green bars, median rsc = 0.486), and negative correlations were more likely to occur among pairs of neurons that responded more strongly to different individual stimuli (‘incongruent,’ brown bars, median rsc = –0.14, p<0.0001, see ‘Methods’). This bimodal distribution did not occur when only a single stimulus was presented (dashed orange line). (d, e) This pattern of results held even when all the unit pairs were considered in aggregate (d, ‘congruent preference’ pairs, median rsc = 0.252; ‘incongruent preference’ pairs, median rsc = –0.052, p<0.0001), and also occurred for well-isolated single units (e). (f) However, among pairs recorded during presentation of superimposed gratings, this pattern was not apparent: unit pairs tended to show positive correlations in both cases (‘congruent-preference’ median rsc = 0.159, ‘incongruent-preference’ median rsc = 0.144), and there was little evident difference compared to when a single grating was presented (orange line). See Figure 4—source data 1 for additional information.
Figure 5.
Figure 5.. Median spike count correlations as a function of congruent-incongruent preference (panel a vs. panel b) and as a function of the spike count response profile classification resulting from the Bayesian model comparison for the V1 adjacent stimulus dataset.
The ‘mixture’-‘mixture’ combinations produced the strongest positive (congruent preference pairs) and strongest negative (incongruent preference pairs) median spike count correlations, but all other combinations also involved positive median correlations for congruent preference pairs and negative median correlations for incongruent preference pairs. See Figure 4—source data 1 for additional information.
Figure 6.
Figure 6.. Activity fluctuations in ‘mixture’ pairs of unit conditions are consistent with a bias toward both units in the pair tending to signal the same stimulus at the same time.
This analysis involved the Pearson’s correlation coefficients computed on assignments scores (ras), which take into account whether the response on combined ‘AB’ stimulus presentations is more ‘A-like’ vs. ‘B-like.’ For two units that share a similar preference (e.g., both respond better to A or both respond better to B), this correlation will have the same sign as the spike count correlation (panel a, green points, positively sloped best-fit line). For two units that prefer different stimuli, this correlation will be opposite in sign to the spike count correlation (panel a, brown points, negatively sloped best-fit line). The overall positive skew in the assignment score correlations for both the ‘same’ and ‘different’ preferring unit condition pairs (panel b) therefore indicates a bias for the same stimulus at the same time. The negative tail indicates the other stimulus is nevertheless also represented in a (smaller) subpopulation of neurons.
Figure 7.
Figure 7.. Results of the spike count response pattern classification analysis for V4 units.
Shown here are classifications for all units regardless of confidence level, and results from gabors and natural images are combined. See Figure 7—figure supplement 1 for a breakdown by confidence level and for gabors and natural images separately. ‘Mixtures’ were seen in both datasets, but ‘intermediates’ were seen primarily in the adjacent-stimulus dataset. These two categories can in principle both contain fluctuating activity, and are grouped here as ‘between’ (i.e., the average response for dual stimuli for these two categories is between the average responses to single stimuli). As with V1, the relative proportions of ‘singles’ vs. ‘outsides’ also differed across these datasets. The combined incidence of these ‘not between’ categories was higher for the superimposed dataset than for the adjacent dataset.
Figure 7—figure supplement 1.
Figure 7—figure supplement 1.. Detailed results of the spike count response pattern classification analysis on V4 units for the adjacent (a–c, e) and superimposed datasets (d, f).
Shading indicates the confidence level of the model categorization as described in Figure 3. Experiments involving adjacent gabors and adjacent images are combined in panels (a) and (e) and broken out separately in panels (b) and (c). Panels (e) and (f) show the sums of the corresponding bars in panels (a) and (d).
Figure 8.
Figure 8.. Like V1, pairs of V4 units show different patterns of spike count correlations when there are two adjacent stimuli vs. when there is one superimposed stimulus, depending on the tuning preferences of the pair.
(a) Mixture–mixture pairs for adjacent stimuli (gabors or images), color coded by whether the two units in the pair shared the same or had different tuning preferences. The ‘congruent preference’ and ‘incongruent preference’ median correlations differed (p<0.002, see ‘Methods’). (b) Similar but including intermediate–intermediate pairs since they too may be fluctuating (median difference p<0.0001). (c) All unit pairs tested with adjacent stimuli, regardless of classification in the modeling analysis (median difference p<0.0001). Orange line shows the results for single-stimulus presentations. (d) Similar to (a) but for superimposed stimuli (median difference not significant). (e) Similar to (c) but for superimposed stimuli (median difference not significant). (f) Comparison of median spike count correlations in the adjacent vs. superimposed datasets, color coded by tuning preference.
Figure 9.
Figure 9.. At the population level, each stimulus appears to be encoded by at least some units on every trial.
(a) The activity of 10 simultaneously recorded V1 units on 18 trials in which a particular combination of two adjacent gratings were presented. The activity of each unit was color coded according to how ‘A-like’ (red) or ‘B-like’ (blue) the responses were on that trial. Only units for which ‘mixture’ was the best descriptor of their response patterns are shown (winning probability >0.5, indicating ‘mixture’ was at least as likely as all other possibilities combined). There are both red and blue squares in every row, supporting the interpretation that these cells exhibited fluctuations across trials. There are also red and blue squares in every column, indicating that on every trial some cells were responding in an ‘A-like’ fashion and others in a ‘B-like’ fashion. (b) Histogram of the number of cells responding in ‘A-like,’ ‘B-like,’ or intermediate levels on each trial (each trace is a separate trial). (c) Similar histogram, but indicating the number of trials in which each cell responded in an ‘A-like,’ ‘B-like,’ or intermediate firing pattern. (d) A simulation of the expected pattern if the observed fluctuations chiefly involved covert fluctuations of attention – cells would be expected to show strong correlations with each other and respond in ‘A-like’ or ‘B-like’ fashion on the same trials. This simulation was constructed by retaining the cell identity and sets of responses observed for each cell, then instituting a strong correlation between them and shuffling the trials in random order. (e) A simulation of the expected pattern if cells were not fluctuating but instead averaging their inputs. This simulation was constructed by assuming that each trial’s response represented a draw from a normal distribution with the same mean as the observed distribution (0.34) and a standard deviation of 0.10.
Author response image 1.
Author response image 1.. This schematic focuses on the “congruent” situation, assuming these involve neurons with adjacentreceptive fields.
Even within this group, variation in eye position would create both positive and negative correlations, and would not be expected to selectively impact dual stimulus trials or congruent vs incongruent pairs of neurons.
Author response image 2.
Author response image 2.. The red and blue curves illustrate two Poisson distributions with rates A and B.
The black curve illustrates draws from a mixture of those two poissons – in this case, there is a 50% chance that the draw is from rate A and a 50% chance from rate B. The green curve illustrates draws from a single Poisson whose rate is equal to A+B. We would classify the green curve as an outside because A+B>max(A,B). An intermediate is not shown here but it would have a shape like the green curve and a peak between the red and blue curves – e.g. Poi((A+B)/2).
Author response image 3.
Author response image 3.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Adelson EH, Movshon JA. Phenomenal coherence of moving gratings. Nature. 1982;300:523–525. doi: 10.1038/300523a0. - DOI - PubMed
    1. Alonso JM, Chen Y. Receptive field. Scholarpedia. 2009;4:5393. doi: 10.4249/scholarpedia.5393. - DOI
    1. Averbeck BB, Lee D. Coding and transmission of information by neural ensembles. Trends in Neurosciences. 2004;27:225–230. doi: 10.1016/j.tins.2004.02.006. - DOI - PubMed
    1. Averbeck BB, Latham PE, Pouget A. Neural correlations, population coding and computation. Nature Reviews. Neuroscience. 2006;7:358–366. doi: 10.1038/nrn1888. - DOI - PubMed
    1. Berger JO, Pericchi LR. The intrinsic bayes factor for model selection and prediction. Journal of the American Statistical Association. 1996;91:109–122. doi: 10.1080/01621459.1996.10476668. - DOI

Publication types