神经科学如何影响人工智能？看DeepMind在NeurIPS2020最新《神经科学人工智能》报告，126页ppt...

来源：专知

Jane Wang是DeepMind神经科学团队的一名研究科学家，研究元强化学习和受神经科学启发的人工智能代理。她的背景是物理、复杂系统、计算和认知神经科学。

Kevin Miller是DeepMind神经科学团队的研究科学家，也是伦敦大学学院的博士后。他目前正在研究如何理解mice和机器的结构化强化学习。

Adam Marblestone是施密特期货创新公司(Schmidt Futures innovation)的研究员，曾是DeepMind的研究科学家，此前他获得了生物物理学博士学位，并在一家脑机接口公司工作。

Where Neuroscience Meets AI

地址：

https://sites.google.com/view/neurips-2020-tutorial-neurosci/home

大脑仍然是唯一已知的真正通用智能系统的例子。对人类和动物认知的研究已经揭晓了一些关键的见解，如并行分布式处理、生物视觉和从奖赏信号中学习的想法，这些都极大影响了人工学习系统的设计。许多人工智能研究人员继续将神经科学视为灵感和洞察力的来源。一个关键的困难是，神经科学是一个广泛的、异质的研究领域，包括一系列令人困惑的子领域。在本教程中，我们将从整体上对神经科学进行广泛的概述，同时重点关注两个领域——计算认知神经科学和电路学习的神经科学——我们认为这两个领域对今天的人工智能研究人员尤其相关。最后，我们将强调几项正在进行的工作，这些工作试图将神经科学领域的见解引入人工智能，反之亦然。

概要：

概述 Introduction / background (15 min)
认知神经科学 Cognitive neuroscience (30 min)
学习电路与机制神经科学， Learning circuits and mechanistic neuroscience (30 min)
交叉最新进展 Recent advancements at the interp (25 min)

https://sites.google.com/view/neurips-2020-tutorial-neurosci/home

参考文献：

Section 1 - Cognitive Neuroscience

Textbooks

Gazzaniga, M., Ivry, R. B., & Mangun, G. R. (2018). Cognitive Neuroscience. W.W. Norton & Company.
O’Reilly, R. C., & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience. MIT Press.
Abbot, L. F., & Dayan, P. (2001). Theoretical neuroscience. MIT Press.

Reviews: Innateness

Zador, A. M. (2019). A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications.
Pinker, S. (2003). The Language Instinct: How the Mind Creates Language. Penguin UK.
Hinton, G. E., & Nowlan, S. J. (1987). How learning can guide evolution. Complex Systems

Reviews: Vision

Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences,
Andersen, R. A., & Buneo, C. A. (2002). Intentional Maps in Posterior Parietal Cortex. Annual Review of Neuroscience .
Whitwell, R. L., Milner, A. D., & Goodale, M. A. (2014). The Two Visual Systems Hypothesis: New Challenges and Insights from Visual form Agnosic Patient DF. Frontiers in Neurology.
DiCarlo, J. J., & Cox, D. D. (2007). Untangling invariant object recognition. Trends in Cognitive Sciences.
Dehaene, S., & Cohen, L. (2011). The unique role of the visual word form area in reading. Trends in Cognitive Sciences.
Kanwisher, N., & Yovel, G. (2006). The fusiform face area: a cortical region specialized for the perception of faces. Phil. Trans. Royal Society of London: B.

Reviews: Memory

Squire, L. R. (2009). The legacy of patient HM for neuroscience. Neuron.
McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review.
O’Reilly, R. C., & Norman, K. A. (2002). Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. Trends in Cognitive Sciences.

Reviews: Planning

Unterrainer, J. M., & Owen, A. M. (2006). Planning and problem solving: from neuropsychology to functional neuroimaging. Journal of Physiology.
Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron
Miller, K. J., & Venditto, S. J. C. (2021). Multi-step planning in the brain. Current Opinion in Behavioral Sciences

Other works cited

Akrami, A., Kopec, C. D., Diamond, M. E., & Brody, C. D. (2018). Posterior parietal cortex represents sensory history and mediates its effects on behaviour. Nature.
Andersen, R. A., Andersen, K. N., Hwang, E. J., & Hauschild, M. (2014). Optic ataxia: from Balint’s syndrome to the parietal reach region. Neuron.
Balaguer, J., Spiers, H., Hassabis, D., & Summerfield, C. (2016). Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network. Neuron.
Chomsky, N. (1976). Reflections on Language. Fontana Press.
Cohen, N. J., & Squire, L. R. (1980). Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that. Science.
Dagher, A., Owen, A. M., Boecker, H., & Brooks, D. J. (1999). Mapping the network for planning: a correlational PET activation study with the Tower of London task. Brain.
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron.
Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. (2015). Model-based choices involve prospective neural activity. Nature Neuroscience.
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Science.
Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature.
Fernando, C., Sygnowski, J., Osindero, S., Wang, J., Schaul, T., Teplyashin, D., Sprechmann, P., Pritzel, A., & Rusu, A. (2018). Meta-learning by the Baldwin effect. Proceedings of the Genetic and Evolutionary Computation Conference.
Gabrieli, J. D., Corkin, S., Mickel, S. F., & Growdon, J. H. (1993). Intact acquisition and long-term retention of mirror-tracing skill in Alzheimer’s disease and in global amnesia. Behavioral Neuroscience.
Goodale, M. A., Milner, A. D., Jakobson, L. S., & Carey, D. P. (1991). A neurological dissociation between perceiving objects and grasping them. Nature.
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., & Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature.
Guo, Z. V., Li, N., Huber, D., Ophir, E., Gutnisky, D., Ting, J. T., Feng, G., & Svoboda, K. (2014). Flow of cortical activity underlying a tactile decision in mice. Neuron.
Holmes, G. (1918). Dsturbances of visual orientation. British Journal of Ophthalmology
Hopcraft, R., Holdo, R., Mwangomo, E., Mduma, S., Thirgood, S., Borner, M., Fryxell, J., Olff, J., & Sinclair, A. (2015). Why are wildebeest the most abundant herbivore in the Serengeti ecosystem? In A. Sinclair, K. Metzger, S. Mduma, & J. Fryxell (Eds.), Serengeti IV: Sustaining biodiversity in a coupled human-natural system. University of Chicago Press.
James, T. W., Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neuroscience
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM.
Kurth-Nelson, Z., Economides, M., Dolan, R. J., & Dayan, P. (2016). Fast Sequences of Non-spatial State Representations in Humans. Neuron.
Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior.
Miller, K. J., Botvinick, M. M., & Brody, C. D. (2017). Dorsal hippocampus contributes to model-based planning. Nature Neuroscience.
Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., & Terazzi, E. (1991). Perception and action in “visual form agnosia.” Brain.
Milner, B., Corkin, S., & Teuber, H.-L. (1968). Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M. Neuropsychologia.
Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J. Neurophysiology.
Naselaris, T., Bassett, D. S., Fletcher, A. K., Kording, K., Kriegeskorte, N., Nienborg, H., Poldrack, R. A., Shohamy, D., & Kay, K. (2018). Cognitive Computational Neuroscience: A New Conference for an Emerging Discipline. Trends in Cognitive Sciences.
Neisser, U. (2014). Cognitive Psychology
Norman, K. A., & O’Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychological Review.
Pflugshaupt, T., Gutbrod, K., Wurtz, P., von Wartburg, R., Nyffeler, T., de Haan, B., Karnath, H.-O., & Mueri, R. M. (2009). About the role of visual field defects in pure alexia. Brain.
Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural Episodic Control. arXiv [cs.LG]
Russell, S., & Norvig, P. (2002). Artificial Intelligence: A Modern Approach
Schapiro, A. C., Turk-Browne, N. B., Botvinick, M. M., & Norman, K. A. (2017). Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning. Phil. Trans. Royal Society of London: B.
Shallice, T. (1982). Specific impairments of planning. Phil. Trans. Royal Society of London: B.
Shen, E. H., Overly, C. C., & Jones, A. R. (2012). The Allen Human Brain Atlas: comprehensive gene expression mapping of the human brain. Trends in Neurosciences
Snyder, L. H., Batista, A. P., & Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex. Nature.
Starck, J. M., Ricklefs, R. E., & Others. (1998). Avian growth and development: evolution within the altricial-precocial spectrum. Oxford University Press.
Steinmetz, N. A., Zatka-Haas, P., Carandini, M., & Harris, K. D. (2019). Distributed coding of choice, action, and engagement across the mouse brain. Nature.
Stringer, C., Pachitariu, M., Steinmetz, N., Carandini, M., & Harris, K. D. (2019). High-dimensional geometry of population responses in visual cortex. Nature.
Szwed, M., Dehaene, S., Kleinschmidt, A., Eger, E., Valabrègue, R., Amadon, A., & Cohen, L. (2011). Specialization for written words over objects in the visual cortex. NeuroImage.
Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006). A cortical region consisting entirely of face-selective cells. Science.
van Opheusden, B., Galbiati, G., Z. Bnaya Li, Y., & Ma, W. J. (2017). Modeling Decision Tree Search in a Two-Player Game. Proceedings of the 39th Annual Meeting of the Cognitive Science Society.
Vesia, M., & Crawford, J. D. (2012). Specialization of reach function in human posterior parietal cortex. Experimental Brain Research.
Vikbladh, O., Meager, M. R., King, J., Blackmon, K., Devinsky, O., Shohamy, D., Burgess, N., & Daw, N. D. (2018). Two Sides of the Same Coin: The Hippocampus as a Common Neural Substrate for Model-Based Planning and Spatial Memory. Neuron
Wada, Y., & Yamamoto, T. (2001). Selective impairment of facial recognition due to a haematoma restricted to the right fusiform and lateral occipital region. Journal of Neurology, Neurosurgery, and Psychiatry.
Wallace, D. J., Greenberg, D. S., Sawinski, J., Rulla, S., Notaro, G., & Kerr, J. N. D. (2013). Rats maintain an overhead binocular field at the expense of constant fusion. Nature.
Williams, S. C. P., & Deisseroth, K. (2013). Optogenetics. PNAS.
Wimmer, R. D., Ian Schmitt, L., Davidson, T. J., Nakajima, M., Deisseroth, K., & Halassa, M. M. (2015). Thalamic control of sensory selection in divided attention. Nature.
Zoccolan, D., Kouh, M., Poggio, T., & DiCarlo, J. J. (2007). Trade-off between object selectivity and tolerance in monkey inferotemporal cortex. J Neuroscience
Zoccolan, D., Oertelt, N., DiCarlo, J. J., & Cox, D. D. (2009). A rodent model for the study of invariant visual object recognition. PNAS

Section 2 - Circuits and Mechanistic Neuroscience

Shepherd, Gordon M. The synaptic organization of the brain. Oxford university press, 2004.
Fee, Michale S., and Jesse H. Goldberg. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198 (2011): 152-170.
Kornfeld, J., Januszewski, M., Schubert, P., Jain, V., Denk, W., & Fee, M. S. (2020). An anatomical substrate of credit assignment in reinforcement learning. BioRxiv.
Murdoch, D., Chen, R. and Goldberg, J.H., 2018. Place preference and vocal learning rely on distinct reinforcers in songbirds. Scientific reports, 8(1), pp.1-9.
Gielow, M.R. and Zaborszky, L., 2017. The input-output relationship of the cholinergic basal forebrain. Cell reports, 18(7), pp.1817-1830.
Chubykin, A.A., Roach, E.B., Bear, M.F. and Shuler, M.G.H., 2013. A cholinergic mechanism for reward timing within primary visual cortex. Neuron, 77(4), pp.723-735.
O'Reilly, R.C., Hazy, T.E., Mollick, J., Mackie, P. and Herd, S., 2014. Goal-driven cognition in the brain: a computational framework. arXiv preprint arXiv:1404.7591.
O'Reilly, R.C., Wyatte, D. and Rohrlich, J., 2014. Learning through time in the thalamocortical loops. arXiv preprint arXiv:1407.3432.
Guerguiev, J., Lillicrap, T.P. and Richards, B.A., 2017. Towards deep learning with segregated dendrites. ELife, 6, p.e22901.
Sacramento, J., Costa, R.P., Bengio, Y. and Senn, W., 2018. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in neural information processing systems (pp. 8721-8732).
Körding, K.P. and König, P., 2001. Supervised and unsupervised learning with two sites of synaptic integration. Journal of computational neuroscience, 11(3), pp.207-215.
Whittington, J.C. and Bogacz, R., 2019. Theories of error back-propagation in the brain. Trends in cognitive sciences, 23(3), pp.235-250.
Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. and Carmena, J.M., 2014. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nature neuroscience, 17(6), pp.807-809.
George, D., Lazaro-Gredilla, M., Lehrach, W., Dedieu, A. and Zhou, G., 2020. A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model. bioRxiv.
Heeger, D.J., 2017. Theory of cortical function. Proceedings of the National Academy of Sciences, 114(8), pp.1773-1782.
Rolls, E.T., 1996. A theory of hippocampal function in memory. Hippocampus, 6(6), pp.601-620.
Pfeiffer, B.E. and Foster, D.J., 2015. Autoassociative dynamics in the generation of sequences of hippocampal place cells. Science, 349(6244), pp.180-183.
Carrillo-Reid, L., Yang, W., Bando, Y., Peterka, D.S. and Yuste, R., 2016. Imprinting and recalling cortical ensembles. Science, 353(6300), pp.691-694.
Müller, M.G., Papadimitriou, C.H., Maass, W. and Legenstein, R., 2020. A model for structured information representation in neural networks of the brain. Eneuro, 7(3).
Abbott, L.F., Bock, D.D., Callaway, E.M., Denk, W., Dulac, C., Fairhall, A.L., Fiete, I., Harris, K.M., Helmstaedter, M., Jain, V. and Kasthuri, N., 2020. The Mind of a Mouse. Cell, 182(6), pp.1372-1376.
Turner, N.L., Macrina, T., Bae, J.A., Yang, R., Wilson, A.M., Schneider-Mizell, C., Lee, K., Lu, R., Wu, J., Bodor, A.L. and Bleckert, A.A., 2020. Multiscale and multimodal reconstruction of cortical structure and function. bioRxiv.
George D, Lehrach W, Kansky K, Lázaro-Gredilla M, Laan C, Marthi B, Lou X, Meng Z, Liu Y, Wang H, Lavin A. A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science. 2017 Dec 8;358(6368).

Section 3 - Recent advancements at the interp

Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23), 8619-8624.
Yamins, D. L., & DiCarlo, J. J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3), 356-365.
Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503(7474), 78-84.
Song, H. F., Yang, G. R., & Wang, X. J. (2017). Reward-based training of recurrent neural networks for cognitive and value-based tasks. Elife, 6, e21492.
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X. J. (2019). Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2), 297-306.
Dezfouli, A., Morris, R., Ramos, F. T., Dayan, P., & Balleine, B. (2018). Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models. In Advances in Neural Information Processing Systems (pp. 4228-4237).
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., ... & Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature Neuroscience, 21(6), 860-868.
Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577(7792), 671-675.
Akrout, M., Wilson, C., Humphreys, P., Lillicrap, T., & Tweed, D. B. (2019). Deep learning without weight transport. In Advances in neural information processing systems (pp. 976-984).
Miconi, T. (2017). Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6, e20899.
Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., & Maass, W. (2018). Long short-term memory and learning-to-learn in networks of spiking neurons. In Advances in Neural Information Processing Systems (pp. 787-797).
Merel, J., Aldarondo, D., Marshall, J., Tassa, Y., Wayne, G., & Ölveczky, B. (2019). Deep neuroethology of a virtual rodent. In International Conference on Learning Representations.
Greydanus, S., Koul, A., Dodge, J., & Fern, A. (2018, July). Visualizing and understanding atari agents. In International Conference on Machine Learning (pp. 1792-1801). PMLR.
Barrett, D. G., Morcos, A. S., & Macke, J. H. (2019). Analyzing biological and artificial neural networks: challenges with opportunities for synergy?. Current opinion in neurobiology, 55, 55-64.
Morcos, A. S., Barrett, D. G., Rabinowitz, N. C., & Botvinick, M. (2018). On the importance of single directions for generalization. In International Conference on Learning Representations.
Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017). Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems (pp. 6076-6085).
Puri, N., Verma, S., Gupta, P., Kayastha, D., Deshmukh, S., Krishnamurthy, B., & Singh, S. (2019, September). Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution. In International Conference on Learning Representations.
Sussillo, D., & Barak, O. (2013). Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural computation, 25(3), 626-649.
Maheswaranathan, N., Williams, A., Golub, M., Ganguli, S., & Sussillo, D. (2019). Universality and individuality in neural dynamics across large populations of recurrent networks. In Advances in neural information processing systems (pp. 15629-15641).