Cooperative thalamocortical circuit mechanism for sensory prediction errors

Mice

All experiments were performed under the UK Animals (Scientific Procedures) Act of 1986 (PPL PD867676F) following UK Home Office approval and local ethical approval by the Sainsbury Wellcome Centre Animal Welfare Ethical Review Body. A total of 105 mice, including 27 C57BL/6J mice, 24 VIP-Cre mice (JAX 010908, Jackson Laboratory; Cre expressed in VIP interneurons), 43 VIP-Cre × Ai14 mice (JAX 010908 and JAX 007914, Jackson Laboratory; tdTomato expressed in VIP interneurons), 7 SOM-Cre mice (JAX 013044, Jackson Laboratory; Cre expressed in SOM interneurons) and 4 SOM-Cre × Ai14 mice (JAX 013044 and JAX 007914, Jackson Laboratory; tdTomato expressed in SOM interneurons) were used in this study. Both female and male mice, at least 7 weeks old at the start of the experiments, were used. Mice were co-housed with littermates in IVC cages, in reversed day–night cycle lighting conditions, with the ambient temperature and humidity set to 23 °C and 56% relative humidity, respectively. Standard environment enrichment was provided in the form of a running wheel, a clear tube and wooden toys.

Surgical procedures

Prior to surgery, Dexadreson (2–3 mg kg^−1) and Carprofen (5 mg kg^−1) were administered. General anaesthesia was induced with 2.5–3% isoflurane, which was then reduced to maintain a breathing rate of around 1 Hz. A 3- or 4-mm craniotomy was made over the right V1, centred on 2.45 mm lateral and 3.6 mm posterior of bregma. For two-photon calcium imaging and optogenetic manipulations of V1 cells, we injected adeno-associated virus (AAV) vectors into right monocular V1 (centred on 2.45 mm lateral and 3.7 mm posterior of bregma, 1–3 injections per mouse, 100–150 nl per injection). For two-photon calcium imaging and optogenetic manipulations of pulvinar axons, we injected AAV vector into the right pulvinar (calcium imaging and optogenetic activation: 1.6 mm lateral and 2.1 mm posterior of bregma, 2.35 below the cortical surface, 1 injection per mouse, 60 nl per injection; optogenetic inactivation: 1.55 mm lateral and 2.0 mm posterior of bregma, 2.3 mm below the cortical surface, 1.60 mm lateral and 2.2 mm posterior of bregma, 2.4 mm below the cortical surface, 2 injections per mouse, 60 nl per injection). All injections were performed using glass pipettes and Nanoject III microinjector (Drummond Scientific) or a pressure injection system (Picospritzer III, Parker). A 3- or 4-mm circular cover glass was glued in place using cyanoacrylate glue (Pattex). A custom-designed stainless steel head plate was attached to the skull using dental cement (Super-Bond C&B, Sun Medical). Animals were given analgesics (Carprofen; 5 mg kg^−1) at 24 and 48 h after surgery. Imaging started approximately 3 weeks after the virus injection.

Viral constructs

We used AAV1-hSyn-GCaMP6f (2 × 10¹³â€‰vg ml^−1 Penn Vector Core/Addgene; diluted 1:8 to 1:15 in saline) for experiments involving two-photon calcium imaging of V1 layer 2/3 cells; AAV1-hSyn-GCaMP7b (2 × 10¹³â€‰vg ml^−1 Penn Vector Core/Addgene; diluted 1:2 in saline) or AAV1-hSyn-axon-GcaMP6s (9 × 10¹²â€‰vg ml^−1 Penn Vector Core/Addgene; diluted 1:2 in saline) for imaging of pulvinar axons; AAV2-EF1a-DIO-eNpHR3.0-mCherry (4.0 × 10¹²â€‰vg ml^−1, 1:2 to 1:10 dilution, UNC vector core) for optogenetic silencing of VIP cells or SOM cells; AAV2-hSyn-eNpHR3.0-mCherry (3.3 × 10¹²â€‰vg ml^−1, 1:2 to 1:4 dilution, UNC vector core) for optogenetic silencing of pulvinar axons; AAV1-hSyn-Flex-ChrimsonR-tdTomato (3.9 × 10¹²â€‰vg ml^−1, 1:2 to 1:5 dilution, UNC vector core) for optogenetic activation of VIP cells; AAV1-Syn-ChrimsonR-tdTomato (4.1 × 10¹²â€‰vg ml^−1, 1:2 to 1:5 dilution, UNC vector core) for optogenetic activation of pulvinar inputs; AAV1-hEF1a-mCherry (5.7 × 10¹²â€‰vg ml^−1, 1:2 to 1:5 dilution, Zurich vector core) for control experiment for LED light stimulation.

Behavioural setup

Behavioural setups consisted of a styrofoam running wheel, two visual stimulation display monitors (see below), a reward delivery spout, and a camera for recording the pupil. Mice were head-fixed and placed on a styrofoam wheel (20 cm diameter, 12 cm width). Their running speed was monitored using a rotary encoder (Kubler Encoder 1000 ppr) coupled to the wheel axle. Reward (a drop of strawberry milk, 50% Ensure nutrition shake, Abbott Laboratories) was delivered by a lick spout in front of the mouse and was regulated via a solenoid pinch valve (161P011, NResearch). Licks were detected with a piezoelectric diaphragm sensor (7BB-12-9, Murata) placed under the spout. Images of the left eye were recorded with a CMOS camera (22BUC03, Imaging Source) at 30 Hz in order to track eye movements and pupil size. The recording of the encoder, presentation of visual stimuli, opening of the reward valves, and camera recordings were controlled by custom-written software in LabView. Behavioural data were acquired using a PCIe 6321 acquisition card (National Instruments).

Food restriction and pre-training

Before mice underwent training in the virtual environment, they were food-restricted and pre-trained to encourage continuous running on the styrofoam wheel. Four to seven days after surgery, food restriction and pre-training started. Mice were weighed daily and given typically 2–3 g of food pellet in addition to strawberry milk given in training sessions to ensure they maintained around 90%, but at least 85%, of their starting body weight. For the first few days, animals were handled in a soft cloth and iteratively fed strawberry milk (Abbott Laboratories) through a syringe until they got used to short manual restraint of the head plate. Mice were then head-fixed and put on the freely rotating styrofoam wheel for 15–60 min. Mice were encouraged to run on the wheel by delivering strawberry milk rewards after they moved a short distance (initially set to ~10 cm). This distance was adjusted (up to 500 cm) depending on the running speed of the mouse, such that mice received roughly one reward every 30 s. Additional rewards were occasionally delivered by the experimenter. This pre-training took 4–10 days.

Virtual corridor

Once mice were running continuously, they were moved to a virtual environment consisting of a linear corridor with varying wall patterns as described previously¹⁴. The cylinder’s rotation (the instantaneous running speed of the animal) was used to control the speed at which the animal moved through the virtual environment. The virtual environment was displayed on two monitors (U2715H, Dell; 60 Hz refresh rate), placed 21 cm away from both eyes of mice and oriented at 35° relative to the midline. Each monitor covered a visual field of approximately 110° horizontally and 60° vertically. All elements of the corridor including the gratings were calibrated to be isoluminant (10.1 cd m^−2). The luminance of the monitor was set at 0.1 cd m^−2, 10.1 cd m^−2 and 20.1 cd m^−2, at black, grey and white values, respectively. The luminance of visual stimuli was measured using a luminance meter (Konica Minolta, LS-100). The grey walls of the virtual corridor were lined with four different landmarks. The last landmark represented the reward zone located at the end of the corridor. Reaching the reward zone triggered an automatic reward delivered by a spout located in front of the mouse. After the reward delivery, the virtual environment was reset to the beginning of the corridor to start the next trial.

Grating stimuli were suddenly presented on full screen once the mouse entered a certain position in the corridor. This was done to ensure precise control of when the mouse would first see the grating. Grating stimuli were presented at four different positions between landmarks. The optic flow of the grating stimuli was ‘uncoupled’ from the running speed for 2.4 s, such that the animal’s locomotion did not affect its temporal frequency. Gratings were square-wave gratings, with the spatial frequency of approximately 0.04 cycles per degree (cpd) at the centre of the monitor and the temporal frequency approximately 2 cycles per second (Hz). Duration of the grating presentation was approximately 2 s at the centre of the monitors. The precise timing of visual stimulus onsets was recorded with a photodiode (Thorlabs) attached to the monitor.

During 5 training sessions, the virtual corridor and the sequence of the four grating stimuli was identical (A–B–A–B) on every trial. In the subsequent imaging session, the identity of one of the four grating stimuli was changed. In block 1 of this session (160 trials), the identity of the 4th grating stimulus B changed either to a novel grating stimulus C (C session), a novel stimulus D (D session), familiar stimulus A (A session) or no stimulus was shown (omission session) on randomly chosen 10% of trials. In block 2, the novel stimulus or no stimulus was shown at the fourth position in 100% of trials. Occasionally one mouse underwent several sessions with unexpected stimuli. In that case, mice went through another training session (with gratings A–B–A–B) in between. For imaging of pulvinar axons, block 1 was shortened to 60 trials, and either a novel grating stimulus C (C session) or a novel stimulus D (D session) was shown at the fourth position on randomly chosen 15% of trials. In block 2, the novel stimulus was shown in 100% of trials, as for imaging of V1 layer 2/3 cells. For experiments in Extended Data Figs. 3a–e and 5l–p, a horizontal grating stimulus E was shown at position 1 and 3 instead of grating stimulus A (E–B–E–B). A novel stimulus C (C session) or a novel stimulus A (A session) was shown on randomly chosen 10% of trials. For experiments in Extended Data Figs. 8 and 9a–c, we used a short version of the virtual corridor with two grating stimuli (A–B) and the identity of the 2nd grating stimulus B changed to familiar stimulus A on randomly chosen 10% of trials.

Visual stimulation

For experiments in Extended Data Fig. 9d–f, visual stimuli were generated using the open-source Psychophysics Toolbox⁶¹ based on MATLAB (MathWorks) and were presented full-field on one monitor at approximately 21 cm from the left eye of the mouse, covering 110° of visual space. Square-wave gratings (spatial frequency: 0.04 cpd, temporal frequency: 2 Hz, duration: 2 s, interval: 4 s, directions: 0 to 360° in 45° increments) were randomized in order and presented 10 times per direction.

Two-photon calcium imaging

Two-photon calcium imaging was performed using a commercial resonance scanning two-photon microscope (B-Scope; Thorlabs) with a 16× water-immersion objective (NA 0.8, Nikon), with a Ti::Sapphire laser at 930 nm excitation wavelength (Mai Tai, SpectraPhysics). Emission light was band-pass filtered using a 525/50 filter for GCaMP and a 607/70 filter for tdTomato/mCherry (Semrock). Images of 512 × 512 pixels from four imaging planes with fields of view ranging from 380 × 380 μm to 440 × 440 μm were acquired at 7.5 Hz frame rate for imaging of V1 neurons and of a single plane of 160 × 160 μm at 15 Hz frame rate for imaging of pulvinar axonal boutons using ScanImage⁶². For imaging of V1 neurons, we used a piezo-actuator (Physik Instrumente) to move the objective in steps of 15 μm between frames to acquire images at four different depths, thus reducing the effective frame rate to 7.5 Hz. Imaging of V1 neurons was performed in layer 2/3 (typically 150–200 μm below the cortical surface). The laser power under the objective never exceeded 35 mW. Axonal bouton calcium measurements were performed in cortical layer 1 (35–55 μm below the cortical surface), with laser powers below 20 mW.

To avoid cross-talk between imaging and visual stimulation, the monitor backlight was controlled using a custom-built circuit to present visual stimuli only at the resonant scanner turnaround points in between two subsequent imaging lines (when data were not acquired)⁶³. The frame trigger signal during two-photon calcium imaging was recorded by Labview and used for synchronization between the calcium imaging frames and task related data (for example, behaviour data and visual stimuli onsets).

For imaging of pulvinar axons, we used VIP-Cre × Ai14 mice. We simultaneously imaged pulvinar axons expressing GCaMP and neurites of VIP neurons expressing tdTomato in layer 1. We then used the red signal (tdTomato) as a structural marker to perform Z-drift correction during imaging and frame registration during data pre-processing.

Optogenetic manipulation

Simultaneous two-photon imaging and optogenetic stimulations were performed as previously described¹⁵. Briefly, 595 nm light was delivered through the objective lens using a fast LED (UHP-T-595, Prizmatix). The LED light power was set to 8 mW in front of the objective. To combine two-photon imaging and optogenetic manipulation, the LED for optogenetic manipulation was synchronized to the resonant scanner turnaround points (when data were not acquired). The propagation of reflected light to the eyes of the mouse was blocked by a metal light shield cone placed on the head plate and a black cement wall around the implant. Optogenetic manipulation occurred in randomly chosen 10–50% of each trial type. For most of optogenetic manipulations, LED stimulation was applied continuously for 3 s, starting at visual stimulus onset. For optogenetic silencing during passive visual stimulation (Extended Data Fig. 9d–f), LED stimulation was applied throughout visual stimulus presentation (2 s). For optogenetic activation (Fig. 4b–d and Extended Data Fig. 11a–c,f–i), LED stimulation was applied at a frequency of 20 Hz, with 40% duty cycle (20 ms pulses) for 1 s starting 0.1 s after visual stimulus onset.

Histology

At the end of each experiment, targeting of virus injections was confirmed by histology. Brains were extracted and fixed overnight in 4% paraformaldehyde, and stored in a 50 mM phosphate buffer. Brains were embedded in 5% agarose and imaged using serial section⁶⁴ two-photon⁶⁵ microscopy. Our microscope was controlled by ScanImage Basic (MBF Bioscience) using BakingTray, a custom software wrapper for setting up the imaging parameters⁶⁶. Images were assembled using StitchIt⁶⁷. Coronal slices were cut at a thickness of 40 μm using a vibratome (Leica VT1000), and imaged every 20 µm with a 16× water-immersion objective (NA 0.8, Nikon). Whole brain coronal image stacks were acquired at a resolution of 4.4 × 4.4 × 20 µm in xyz, with a two-photon laser wavelength of 780 nm, and approximately 130 mW at the sample. Selected brain images were registered to the adult mouse Allen common coordinate framework⁶⁸ using The Slice Histology Alignment, Registration, and Probe-Track analysis (SHARP-Track), a MATLAB based registration pipeline with optimized parameters for mouse brain registration at various cutting angles⁶⁹. A subset of brains was embedded in 4% agarose (A9539, Sigma), cut in 200 μm coronal slices on a vibratome (HM650V; Microm), mounted in a mounting medium containing DAPI (Vectashield; Vector Laboratories) and imaged on a slide scanner (Zeiss AxioScan) or on a confocal microscope (Leica SP8).

Quantification and statistical analysis

Two-photon imaging

Two-photon imaging frames were motion corrected and segmented using custom-written scripts in MATLAB as previously described³¹. In brief, to correct for x–y motion, two-photon imaging frames were registered to a 1,200-frame average (40 frames × 30 batches) using a phase-correlation algorithm. When the same V1 neurons were imaged over multiple sessions, images from those sessions were registered together, and identical cells were matched across sessions by using custom-written software. Frames with large motion were detected by inspecting the registration displacement results and were discarded from further analysis. Regions of interest (ROIs) were detected semi-automatically using intensity thresholding combined with principal component analysis–independent component analysis refinement and validated and refined manually. All time series were extracted and analysed with custom-written functions using the TimeSeriesAnalysis package⁷⁰. All pixels within each ROI were averaged to give a single time course. Contaminating signals from neuropil were subtracted using an asymmetric Student’s t model (ast_model; https://github.com/BaselLaserMouse/ast_model). Calcium ΔF/F₀ signals were obtained by using the baseline fluorescence F₀, which is estimated by a Gaussian mixture model with two components fitted on the raw fluorescence data. The mean parameter of the lowest Gaussian component is used as F₀. To be able to compare calcium activity across sessions and mice, z-scored ΔF/F was computed by subtracting the mean value of ΔF/F of a session and dividing the resulting trace by the standard deviation.

Analysis of visual responses

The response to each grating was calculated using the mean z-scored ΔF/F calcium signal averaged over a window from 0.4 s to 2 s after grating onset, baseline-subtracted using the mean z-scored ΔF/F signal during 0.5 s before stimulus onset for each grating presentation. Neurons were classified as stimulus-responsive if their mean response was bigger than 0.5 z-scored ΔF/F. In Fig. 1, cell-averaged calcium traces are from neurons responsive to the presented grating in trials with unexpected C or D (block 1), trials with expected C or D trials (late block 2) or both. For comparison, Extended Data Fig. 2 shows cell-averaged calcium responses of all neurons responsive to any grating. In Fig. 2a,b and Extended Data Figs. 5u–z and 6a,b, cells were defined as prediction-error-responsive if the responses were significantly different between unexpected stimuli C4, D4 or stimulus omission (block 1) and expected stimuli C4, D4 or stimulus omission (second half of block 2, two-sided t-test; α = 0.05; unexpected C4, D4 or omission versus expected C4, D4 or omission) and the difference in response was larger than 0.5 z-scored ΔF/F. Similarly, in Extended Data Figs. 5e,p and 7e, cells were defined as prediction-error-responsive if the responses were significantly different between unexpected C4 or D4 (block 1) and expected C4 or D4 (second half of block 2, two-sided t-test; α = 0.05; unexpected C4 or D4 versus expected C4 or D4) and the difference in response was larger than 0.3 z-scored ΔF/F. In Fig. 3f,g,m,n, average response in LED on and off trials was used for classification of stimulus-responsive cells to avoid selection bias towards LED off trials. In Fig. 4, response in LED off trials was used for classification of stimulus-selective cells to avoid inclusion of opsin-expressing, therefore directly activated VIP cells. In Fig. 5 and Extended Data Fig. 12, SOM cells or VIP cells were defined as ‘recruited’ if their responses were significantly different between with and without optogenetic pulvinar axon stimulation (two-sided t-test; α = 0.016; with versus without LED light stimulation) and the difference in response was larger than 0.3 z-scored ΔF/F, during at least one of the visual stimulus presentations (expected B4, unexpected C4 or D4, expected C4 or D4). In Extended Data Fig. 5f–k, cells were defined as prediction-error (C or D) responsive but not responsive to expected C or D if the responses were significantly different between unexpected C4 or D4 and expected C4 or D4 (two-sided t-test; α = 0.05) and the difference in response was larger than 0.5 z-scored ΔF/F, but the response to expected C4 or D4 was smaller than 0.5 z-scored ΔF/F. In Extended Data Fig. 6g,h, in which raw ΔF/F rather than z-scored ΔF/F was used, neurons were defined as stimulus-responsive if their stimulus response strength was larger than 0.2 ΔF/F in the second half of block 2. In Extended Data Fig. 7j,k, boutons were defined as stimulus-responsive if the response to any expected stimulus in late block 2 was larger than 0.1 z-scored ΔF/F.

Selectivity and selectivity index

To quantify the selectivity of neural responses we computed a response selectivity measure for individual V1 layer 2/3 cells and pulvinar boutons:

Where R_{C4 or D4} is the mean response to the gratings C4 or D4 in late block 2, and R_{A3 or B2} is the mean response to the gratings A3 or B2 in late block 2. Selectivity values of >1 or <−1 were shown as 1 or −1, respectively. If selectivity of neurons responsive to a specific visual stimulus was less than 0.6 or more than 0.8, they were classified as either non-selective or highly selective to that stimulus, respectively. We also used an additional selectivity index (SI) to quantify response selectivity of individual pulvinar boutons (Extended Data Fig. 7l–n), since this index provided a more reliable measure for the noisy bouton calcium traces. SI was calculated as previously described⁷¹. In brief, it was computed from the difference between the mean response to the expected stimulus C4 or D4 and expected stimulus B2 in late block 2, divided by the pooled standard deviation of the responses.

Fast and slow running trials

For the analysis in Extended Data Fig. 1k,l, trials in block 1 and 2 were divided into fast and slow running trials based on mean running speed during presentation of grating C4. A time window starting 0.4 s and ending 2 s after the grating onset, similar to the response window used for calcium responses, was used to calculate the mean running speed. A trial was defined as ‘fast’ or ‘slow’ if the mean running speed during the time window was in the top 50th or bottom 50th percentile of all visual stimulus C4 presentations in block 1 or 2.

Correlation of running speed and neuronal activity

To determine the effect of running speed on neuronal activity (Extended Data Fig. 1m,n), we computed for each cell the correlation between mean ΔF/F and mean running speed in a time window (starting 0.4 s and ending 2 s after grating stimulus onset) of each trial in block 1 or 2. For the analysis in Extended Data Fig. 1o, we used the square of the correlation coefficient (R², coefficient of determination) of running speed and ΔF/F across the recording, to quantify the strength of the modulation of neural responses by running speed across the entire session (block 1 and 2).

Pupil size

Pupil size was computed offline. The pupil was detected using a binary threshold and centre of mass of the detected regions. We then applied a one-dimensional filter to the traces using the filloutlier function in MATLAB.

Statistics

We used two-sided Wilcoxon signed-rank tests for comparisons across animals and hierarchical bootstrapping test for comparisons across cells unless otherwise stated. Hierarchical bootstrap procedure was performed as previously described^72,73. In short, we first randomly resampled animals with replacement and then resampled cells with replacement from each of the resampled animals. We then randomly shuffled the paired data and calculated the statistic of interest. This process was repeated 10,000 times. The statistic values were compared against the value of the original data to calculate P values. Where relevant P values were adjusted for multiple comparisons using Bonferroni correction, as indicated in the figure legends. For the randomization test, we computed the statistic of interest with randomly shuffled data (10,000 times). The statistic values were compared against the value of the original data to calculate P values. All tests were performed using MATLAB. Mean and bootstrap 95% confidence intervals were used for display purposes, as stated in the figure legends. Confidence intervals were estimated using bootci function in MATLAB, with 10,000 bootstrap samples with replacement. Cohen’s d was computed from the difference between the two mean responses, divided by the pooled standard deviation of the responses. No statistical methods were used to predetermine sample sizes, but our sample sizes are similar to those generally used in the field. Experimenters were not blinded to experimental groups. Animals were allocated to experimental groups pseudo-randomly, and trial types (expected or unexpected stimuli, with or without optogenetic manipulation) were randomly interleaved.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.