Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Gershman, S. J. & Uchida, N. Believing in dopamine. Nat. Rev. Neurosci. https://doi.org/10.1038/s41583-019-0220-7 (2019).
Wood, A. N. New roles for dopamine in motor skill acquisition: lessons from primates, rodents, and songbirds. J. Neurophysiol. 125, 2361–2374 (2021).
Blain, B. & Sharot, T. Intrinsic reward: potential cognitive and neural mechanisms. Curr. Opin. Behav. Sci. 39, 113–118 (2021).
Hisey, E., Kearney, M. G. & Mooney, R. A common neural circuit mechanism for internally guided and externally reinforced forms of motor learning. Nat. Neurosci. 21, 589–597 (2018).
Thorndike, E. L. The Elements of Psychology (Seiler, 1905).
Markowitz, J. E. et al. Spontaneous behaviour is structured by reinforcement without explicit reward. Nature 614, 108–117 (2023).
Doupe, A. J. & Kuhl, P. K. Birdsong and human speech: common themes and mechanisms. Annu. Rev. Neurosci. 22, 567–631 (1999).
Sakata, J. T., Woolley, S. C., Fay, R. R. & Popper, A. N. The Neuroethology of Birdsong (Springer Nature, 2020).
Mooney, R. Birdsong. Curr. Biol. 32, R1090–R1094 (2022).
Derégnaucourt, S., Mitra, P. P., Fehér, O., Pytte, C. & Tchernichovski, O. How sleep affects the developmental learning of bird song. Nature 433, 710–716 (2005).
Goffinet, J., Brudner, S., Mooney, R. & Pearson, J. Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires. eLife 10, e67855 (2021).
Brudner, S., Pearson, J. & Mooney, R. Generative models of birdsong learning link circadian fluctuations in song variability to changes in performance. PLoS Comput. Biol. 19, e1011051 (2023).
Eales, L. A. Song learning in zebra finches: some effects of song model availability on what is learnt and when. Anim. Behav. 33, 1293–1300 (1985).
Price, P. H. Developmental determinants of structure in zebra finch song. J. Comp. Physiol. Psychol. 93, 260–277 (1979).
Singh Alvarado, J. et al. Neural dynamics underlying birdsong practice and performance. Nature 599, 635–639 (2021).
Person, A. L., Gale, S. D., Farries, M. A. & Perkel, D. J. Organization of the songbird basal ganglia, including area X. J. Comp. Neurol. 508, 840–866 (2008).
Feenders, G. et al. Molecular mapping of movement-associated areas in the avian brain: a motor theory for vocal learning origin. PLoS ONE 3, e1768 (2008).
Doya, K. & Sejnowski, T. A novel reinforcement model of birdsong vocalization learning. In Proc. Advances in Neural Information Processing Systems Vol. 7 (eds Tesauro, G. et al.) 101–108 (MIT, 1994).
Fee, M. S. & Goldberg, J. H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198, 152–170 (2011).
Duffy, A., Latimer, K. W., Goldberg, J. H., Fairhall, A. L. & Gadagkar, V. Dopamine neurons evaluate natural fluctuations in performance quality. Cell Rep 38, 110574 (2022).
Gadagkar, V. et al. Dopamine neurons encode performance error in singing birds. Science 354, 1278–1282 (2016).
Xiao, L. et al. A basal ganglia circuit sufficient to guide birdsong learning. Neuron 98, 208–221.e5 (2018).
Mohebi, A., Collins, V. L. & Berke, J. D. Accumbens cholinergic interneurons dynamically promote dopamine release and enable motivation. eLife 12, e85011 (2023).
Liu, C. et al. An action potential initiation mechanism in distal axons for the control of dopamine release. Science https://doi.org/10.1126/science.abn0532 (2022).
Kramer, P. F. et al. Synaptic-like axo-axonal transmission from striatal cholinergic interneurons onto dopaminergic fibers. Neuron 110, 2949–2960.e4 (2022).
Sun, F. et al. Next-generation GRAB sensors for monitoring dopaminergic activity in vivo. Nat. Methods 17, 1156–1166 (2020).
Ko, D. & Wanat, M. J. Phasic dopamine transmission reflects initiation vigor and exerted effort in an action- and region-specific manner. J. Neurosci. 36, 2202–2211 (2016).
Panigrahi, B. et al. Dopamine is required for the neural representation and control of movement vigor. Cell 162, 1418–1430 (2015).
Roeser, A. et al. Dopaminergic error signals retune to social feedback during courtship. Nature 623, 375–380 (2023).
Bottjer, S. W., Halsema, K. A., Brown, S. A. & Miesner, E. A. Axonal connections of a forebrain nucleus involved with vocal learning in zebra finches. J. Comp. Neurol. 279, 312–326 (1989).
Chantranupong, L. et al. Dopamine and glutamate regulate striatal acetylcholine in decision-making. Nature 621, 577–585 (2023).
Krok, A. C. et al. Intrinsic dopamine and acetylcholine dynamics in the striatum of mice. Nature 621, 543–549 (2023).
Jing, M. et al. An optimized acetylcholine sensor for monitoring in vivo cholinergic activity. Nat. Methods 17, 1139–1146 (2020).
Zingg, B. et al. AAV-mediated anterograde transsynaptic tagging: mapping corticocollicular input-defined neural pathways for defense behaviors. Neuron 93, 33–47 (2017).
Kozhevnikov, A. A. & Fee, M. S. Singing-related activity of identified HVC neurons in the zebra finch. J. Neurophysiol. 97, 4271–4283 (2007).
Goldberg, J. H. & Fee, M. S. Singing-related neural activity distinguishes four classes of putative striatal neurons in the songbird basal ganglia. J. Neurophysiol. 103, 2002–2014 (2010).
Tumer, E. C. & Brainard, M. S. Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong. Nature 450, 1240–1244 (2007).
Fiete, I. R., Fee, M. S. & Seung, H. S. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038–2057 (2007).
Farries, M. A. & Fairhall, A. L. Reinforcement learning with modulated spike timing dependent synaptic plasticity. J. Neurophysiol. 98, 3648–3665 (2007).
Long, M. A. & Fee, M. S. Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature 456, 189–194 (2008).
Ding, L. & Perkel, D. J. Dopamine modulates excitability of spiny neurons in the avian basal ganglia. J. Neurosci. 22, 5210–5218 (2002).
Kubikova, L., Wada, K. & Jarvis, E. D. Dopamine receptors in a songbird brain. J. Comp. Neurol. 518, 741–769 (2010).
Richfield, E. K., Penney, J. B. & Young, A. B. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30, 767–777 (1989).
Kearney, M. G., Warren, T. L., Hisey, E., Qi, J. & Mooney, R. Discrete evaluative and premotor circuits enable vocal learning in songbirds. Neuron 104, 559–575.e6 (2019).
Hamaguchi, K., Tschida, K. A., Yoon, I., Donald, B. R. & Mooney, R. Auditory synapses to song premotor neurons are gated off during vocalization in zebra finches. eLife 3, e01833 (2014).
Hamaguchi, K. & Mooney, R. Recurrent interactions between the input and output of a songbird cortico-basal ganglia pathway are implicated in vocal sequence variability. J. Neurosci. 32, 11671–11687 (2012).
Dan Foresee, F. & Hagan, M. T. Gauss-Newton approximation to Bayesian learning. In Proc. International Conference on Neural Networks 1930–1935 (IEEE, 1997).
Bates, D., Machler, M., Bolker, B. & Walker, S. C. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2014).
Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Series B Stat. Methodol. 58, 267–288 (1996).
Friedman, J., Hastie, T. & Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33, 1–22 (2010).
Lopes, G. et al. Bonsai: an event-based framework for processing and controlling data streams. Front. Neuroinform. 9, 7 (2015).
Pennington, Z. T. et al. ezTrack: an open-source video analysis pipeline for the investigation of animal behavior. Sci. Rep. 9, 19979 (2019).
