Friday, March 14, 2025
No menu items!
HomeNatureNatural behaviour is learned through dopamine-mediated reinforcement

Natural behaviour is learned through dopamine-mediated reinforcement

  • Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Gadagkar, V. et al. Dopamine neurons encode performance error in singing birds. Science 354, 1278–1282 (2016).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Zhuo, Y. et al. Improved green and red GRAB sensors for monitoring dopaminergic activity in vivo. Nat. Methods 21, 680–691 (2024).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Chen, R. & Goldberg, J. H. Actor–critic reinforcement learning in the songbird. Curr. Opin. Neurobiol. 65, 1–9 (2020).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Joel, D., Niv, Y. & Ruppin, E. Actor–critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw. 15, 535–547 (2002).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).

  • Botvinick, M. et al. Reinforcement learning, fast and slow. Trends Cogn. Sci. 23, 408–422 (2019).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Wickens, J. R., Reynolds, J. N. & Hyland, B. I. Neural mechanisms of reward-related motor learning. Curr. Opin. Neurobiol. 13, 685–690 (2003).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Costa, R. M. Plastic corticostriatal circuits for action learning: what’s dopamine got to do with it? Ann. N. Y. Acad. Sci. 1104, 172–191 (2007).

    Article 
    ADS 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Jarvis, E. Vocal learning and spoken language. Science 366, 50–54 (2019).

    Article 
    ADS 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Davenport, M. H. & Jarvis, E. D. Birdsong neuroscience and the evolutionary substrates of learned vocalization. Trends Neurosci. 46, 97–99 (2023).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Konopka, G. & Roberts, T. F. Insights into the neural and genetic basis of vocal communication. Cell 164, 1269–1276 (2016).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Prather, J., Okanoya, K. & Bolhuis, J. J. Brains for birds and babies: neural parallels between birdsong and speech acquisition. Neurosci. Biobehav. Rev. 81, 225–237 (2017).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Doupe, A. J. & Kuhl, P. K. Birdsong and human speech: common themes and mechanisms. Annu. Rev. Neurosci. 22, 567–631 (1999).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Brainard, M. S. & Doupe, A. J. Translating birdsong: songbirds as a model for basic and applied medical research. Annu. Rev. Neurosci. 36, 489–517 (2013).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Burke, J. E. & Schmidt, M. F. Neural control of birdsong. eLS 1, 345–355 (2020).

    MATH 

    Google Scholar
     

  • Person, A. L., Gale, S. D., Farries, M. A. & Perkel, D. J. Organization of the songbird basal ganglia, including area X. J. Comp. Neurol. 508, 840–866 (2008).

    Article 
    PubMed 

    Google Scholar
     

  • Lovell, P. V. et al. ZEBrA: Zebra finch Expression Brain Atlas—a resource for comparative molecular neuroanatomy and brain evolution studies. J. Comp. Neurol. 528, 2099–2131 (2020).

    Article 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Tumer, E. C. & Brainard, M. S. Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong. Nature 450, 1240–1244 (2007).

    Article 
    ADS 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Andalman, A. S. & Fee, M. S. A basal ganglia–forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl Acad. Sci. USA 106, 12518–12523 (2009).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Duffy, A., Latimer, K. W., Goldberg, J. H., Fairhall, A. L. & Gadagkar, V. Dopamine neurons evaluate natural fluctuations in performance quality. Cell Rep. 38, 110574 (2022).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Roeser, A. et al. Dopaminergic error signals retune to social feedback during courtship. Nature 623, 375–380 (2023).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Hisey, E., Kearney, M. G. & Mooney, R. A common neural circuit mechanism for internally guided and externally reinforced forms of motor learning. Nat. Neurosci. 21, 589–597 (2018).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Xiao, L. et al. A basal ganglia circuit sufficient to guide birdsong learning. Neuron 98, 208–221 (2018).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Hoffmann, L. A., Saravanan, V., Wood, A. N., He, L. & Sober, S. J. Dopaminergic contributions to vocal learning. J. Neurosci. 36, 2176–2189 (2016).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Fee, M. S. & Goldberg, J. H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198, 152–170 (2011).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Mackevicius, E. L. & Fee, M. S. Building a state space for song learning. Curr. Opin. Neurobiol. 49, 59–68 (2018).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Doya, K. & Sejnowski, T. A novel reinforcement model of birdsong vocalization learning. In Adv. Neural Information Processing Systems 7 (NIPS 7) (eds Tesauro, G. et al.) 101–108 (MIT Press, 1995).

  • Tchernichovski, O., Mitra, P. P., Lints, T. & Nottebohm, F. Dynamics of the vocal imitation process: how a zebra finch learns its song. Science 291, 2564–2569 (2001).

    Article 
    ADS 
    PubMed 
    CAS 

    Google Scholar
     

  • Kollmorgen, S., Hahnloser, R. H. R. & Mante, V. Nearest neighbours reveal fast and slow components of motor learning. Nature 577, 526–530 (2020).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Funabiki, Y. & Konishi, M. Long memory in song learning by zebra finches. J. Neurosci. 23, 6928–6935 (2003).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Steinfath, E., Palacios-Munoz, A., Rottschafer, J. R., Yuezak, D. & Clemens, J. Fast and accurate annotation of acoustic signals with deep neural networks. eLife 10, e68837 (2021).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Lerner, T. N., Holloway, A. L. & Seiler, J. L. Dopamine, updated: reward prediction error and beyond. Curr. Opin. Neurobiol. 67, 123–130 (2021).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Toutounji, H., Zai, A. T., Tchernichovski, O., Hahnloser, R. H. R. & Lipkind, D. Learning the sound inventory of a complex vocal skill via an intrinsic reward. Sci. Adv. 10, eadj3824 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Bayer, H. M. & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Adam, I. et al. Daily vocal exercise is necessary for peak performance singing in a songbird. Nat. Commun. 14, 7787 (2023).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Fiete, I. R., Fee, M. S. & Seung, H. S. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038–2057 (2007).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Ikeda, M. Z., Trusel, M. & Roberts, T. F. Memory circuits for vocal imitation. Curr. Opin. Neurobiol. 60, 37–46 (2019).

    Article 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Louder, M. I. M. et al. Transient sensorimotor projections in the developmental song learning period. Cell Rep. 43, 114196 (2024).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Tian, J. et al. Distributed and mixed information in monosynaptic inputs to dopamine neurons. Neuron 91, 1374–1389 (2016).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Watabe-Uchida, M., Eshel, N. & Uchida, N. Neural circuitry of reward prediction error. Annu. Rev. Neurosci. 40, 373–394 (2017).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Chen, R. et al. Songbird ventral pallidum sends diverse performance error signals to dopaminergic midbrain. Neuron 103, 266–276 (2019).

    Article 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Kearney, M. G., Warren, T. L., Hisey, E., Qi, J. & Mooney, R. Discrete evaluative and premotor circuits enable vocal learning in songbirds. Neuron 104, 559–575 (2019).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Bottjer, S. W., Brady, J. D. & Cribbs, B. Connections of a motor cortical region in zebra finches: relation to pathways for vocal learning. J. Comp. Neurol. 420, 244–260 (2000).

    Article 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Mandelblat-Cerf, Y., Las, L., Denisenko, N. & Fee, M. S. A role for descending auditory cortical projections in songbird vocal learning. eLife 3, e02152 (2014).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Schrittwieser, J. et al. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature 588, 604–609 (2020).

    Article 
    ADS 
    PubMed 
    MATH 
    CAS 

    Google Scholar
     

  • Fawzi, A. et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature 610, 47–53 (2022).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Markowitz, J. E. et al. Spontaneous behaviour is structured by reinforcement without explicit reward. Nature 614, 108–117 (2023).

    Article 
    ADS 
    PubMed 
    PubMed Central 
    MATH 
    CAS 

    Google Scholar
     

  • Colquitt, B. M., Merullo, D. P., Konopka, G., Roberts, T. F. & Brainard, M. S. Cellular transcriptomics reveals evolutionary identities of songbird vocal circuits. Science 371, eabd9704 (2021).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • Pfenning, A. R. et al. Convergent transcriptional specializations in the brains of humans and song-learning birds. Science 346, 1256846 (2014).

    Article 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Tchernichovski, O., Nottebohm, F., Ho, C. E., Pesaran, B. & Mitra, P. P. A procedure for an automated measurement of song similarity. Anim. Behav. 59, 1167–1176 (2000).

    Article 
    PubMed 
    CAS 

    Google Scholar
     

  • Immelman, K. in Bird Vocalizations (ed. Hinde, R. A.) 64–74 (Cambridge Univ. Press, 1969).

  • Krzanowski, W. J. Principles of Multivariate Analysis: A User’s Perspective (Oxford Univ. Press, 1988).

  • Seber, G. A. F. Multivariate Observations (Wiley, 1984).

  • Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716–723 (1974).

    Article 
    ADS 
    MathSciNet 
    MATH 

    Google Scholar
     

  • Ljung, L. System Identification: Theory for the User (Prentice Hall, 1999).

  • Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979).

    MathSciNet 
    MATH 

    Google Scholar
     

  • RELATED ARTICLES

    Most Popular

    Recent Comments