In 2016, Prof. Dr Cyriel Pennartz, head of the research group Cognitive and Systems Neuroscience, has been awarded a coordinating role in the Human Brain Project (HBP), an EU-funded Future and Emerging Technologies (FET) Flagship Initiative designed to help advance neuroscience, medicine and computing. With funding amounting to a total of about EUR 0.75 billion, the 10-year project is the biggest European research project in the history of neuroscience. Leading scientists from different disciplines and more than 100 universities and research centers across Europe are collaborating on this unprecedented endeavour. They gather experimental data, which inform theoretical models and are used to create simulations of the brain and its cognitive processes in the EBRAINS Research Infrastructure.
The human brain project consists of nine work packages. Prof. Dr. Cyriel Pennartz coordinated the Understanding Cognition branch in which he and 17 other research groups within the EU worked in the neural mechanisms underlying cognitive processes, such as sleep, memory and consciousness. In the last years of HBP, we are co-leading the coordination of Work Package 2: ‘Networks underlying brain cognition and consciousness’. We participate in different tasks & research lines in HBP, as described below.
The goal is to develop computational models of object recognition in which different sensory modalities –such as vision and touch –converge into a unified representation generated by our brain. We approach this goal by (i) developing “top-down” computational models informed by cognitive and behavioural information, able to identify objects using visual and tactile inputs, under different viewing angles and motor manipulations, (ii) developing “bottom-up” computational models constrained by anatomical and electrophysiological data, able to simulate single and multi-unit neural activity as well as local-field potentials linked to the task, (iii) building a theoretical predictive coding framework which unifies these two approaches to provide predictions of multisensory object recognition, (iv) comparing the predictions of these models with experimental data from mouse and human experiments and involving different perceptual tasks across brain states, and (v) testing these models in neuromorphic hardware and by incorporating them in robots to drive their spatial navigation and recognition of objects and places.
UvA collaborators in this research line are: Matthias Brucklacher, Eric Dijkema, Giulia Moreni, Kwangjun Lee, Jorge Mejias, Umberto Olcese, Thijs Ruikes, Julien Fiorilli, Reinder Dorman, Cyriel Pennartz. (see our staff list for contact details)
Why are some sensory stimuli consciously detected, whereas others remain subliminal? What is the relation between consciousness and cognition? This work is based on previous HBP research. Our theory characterizes conscious experience as a multimodally rich, spatially encompassing representation of one’s world, including one’s own body. Consciousness has a clearly outlined, biological function: presenting the subject with an overview of its personal situation in order to facilitate and enable complex decision-making and planned, goal-directed behavior. This behavior is kept apart from reflexes and habits, which can proceed largely unconsciously. Representations are built in cortical hierarchies marked by both feedforward and feedback trafficking of information, a process resembling predictive coding approaches to sensory processing. Based on sensory inputs, higher cortical areas generate a ‘best-guess’ representation of what causes of the inputs. However, by itself predictive coding is not deemed sufficient to reach consciousness. A multi-level representational scheme is proposed to transition from low-level sensory representations, via unimodal networks, to multimodal networks coding conscious representations. Next to feedforward and feedback interactions, lateral (e.g. cross-modal) connectivity in the cortex plays a key role in conceptualizing how multimodal representations emerge from low-level network computations. This theoretical work is followed up with experiments on neural mechanisms underlying (loss of) consciousness and has strong consequences for our thinking about awareness in animals, AI and intelligent robots. In 2021, Pennartz published a popular-science account on consciousness, drawing together knowledge from medicine, neuroscience, philosophy and psychology and a bit of science history (in Dutch; The Code of Consciousness).
Collaborators in this research line are: Cyriel Pennartz, Lilian Emming, Mathis Bassler (for contact details, see our staff list)
External collaborators: Michele Farisco (Uppsala University), Kathinka Evers (Uppsala University)
Facility Hubs open up the perspective for users from academia and industry Europe-wide to use the most advanced, cutting edge facilities and resources of HBP partners for their own research projects, to start collaborations in this field and to carry out beyond state-of-the-art science.
At the Cognitive and Systems Neuroscience department, we harbour one of the largest groups for 2-photon in vivo neuronal imaging and ensemble recordings in Europe, which in the context of EBRAINS will function as a competence center in its field of expertise. Two-photon imaging and ensemble recordings, both usually conducted in behaving, task-performing animal models, serve a common purpose within the field: to understand how cognitive, perceptual and motor functions originate from the behavior of large, cooperating populations of neurons. The goal is to host up to 8 medium/long and 8 short projects per year at PhD, postdoc or staff level (undergraduate projects are taken to contribute to these). Such projects will often become part of a larger collaborative publication somewhat later. Further dissemination will occur via conferences, demonstrator projects, summer schools, symposia, public awareness events. The Hub will generate indirect impact when other labs across the world adopt the model of “chaining” a facility hub with computational facilities for further data processing, storage and modelling.
Collaborators in this task are: Angelica da Silva Lantyer, Paul Mertens, Cyriel Pennartz, Lilian, Mathis Bassler, Umberto Olcese, Conrado Bosman (for contact details, see our staff list).
External collaborators: Martin Pearson (University of the West England Bristol), Shirin Dora (Ulster University), Sander Bohte (Centrum Wiskunde & Informatica), Pieter Roelfsema (NIN Amsterdam), Walter Senn (University of Bern), Giovanni Pezzulo (CNR Rome), Emrah Duzel (DZNE Magdeburg).
This task emerges from the need of ensuring the successful interaction of all scientists in WP2 with EBRAINS by means of providing to them the required technical guidance and support, supervision of goal achievement and scientific integration. This Technical Task is not only a technical support task, but it vertebrates that the scientific developments in WP2 are properly integrated and available in EBRAINS through curated dataset collections, analysis pipelines and models.
The Task is also pivotal for the implementation of the technical aspects of the Showcases and for the integration in EBRAINS of WP2 Use-Cases and Live papers. T2.11 is in permanent contact with the EBRAINS Technical Coordination for the two major following aspects: 1) provide feedback and report any issue from WP2 partners to EBRAINS and 2) communicate to WP2 partners the new updates and developments that are occurring in EBRAINS.
Collaborator in this task: Angelica da Silva Lantyer
Fiorilli, J.P.N., Bos, J.J., Grande, X., Lim, J., Düzel, E. & Pennartz, C. M. A. (2021). Reconciling the object and spatial processing views of the Perirhinal Cortex through task-relevant unitization. Hippocampus. http://dx.doi.org/10.1002/hipo.23304
Meijer, G.T., Marchesi, P., Mejias, J.F., Montijn, J.S., Lansink, C. S., Pennartz, C.M.A. (2020). Neural Correlates of Multisensory Detection Behavior: Comparison of Primary and Higher-Order Visual Cortex. Cell Reports, Vol. 21 No. 6. http://dx.doi.org/10.1016/j.celrep.2020.107636
Arnts, H., van Erp, W. S., Boon, L.I., Bosman, C.A., Admiraal, M.M., Schrantee, A., Pennartz, C.M.A., Schuurman, R., Stam, C.J., van Rootselaar, A.-F., Hillebrand, A., van den Munckhof, P. (2020). Awakening after a sleeping pill: Restoring functional brain networks after severe brain injury. Cortex, Vol. 132. http://dx.doi.org/10.1016/j.cortex.2020.08.011
Rusu, S. I., & Pennartz, C. M. A. (2020). Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems. Hippocampus, 30(1), 73-98. https://doi.org/10.1002/hipo.23167
Pennartz CMA, Farisco M, Evers K (2019) Indicators and criteria of consciousness in animals and intelligent machines: an inside-out approach. Frontiers in Systems Neuroscience Vol. 13, https://doi.org/10.3389/fnsys.2019.00025 .
Pennartz CMA, Dora S, Muckli L, Lorteije JAM (2019) Towards a unified view on pathways and functions of neural recurrent processing. Trends in Neurosciences 42: 589-603. https://doi.org/10.1016 /j.tins.2019.07.005 .
Bos JJ, Vinck M, Marchesi P*, Keestra, A, Van Mourik-Donga LA, Jackson JC, Verschure PFMJ, Pennartz CMA (2019) Multiplexing of information about Self and Others in Hippocampal Ensembles. Cell Reports 29: 1-13. http://doi.org/10.1016/jcelrep.2019.11.057.
Meijer GT, Olcese, Mertens PEC, Pennartz CMA, Olcese U, Lansink CS (2019) Cortical networks for multisensory processing: distinct functions sharing a common circuitry. Progr. Neurobiology 174: 1-15.
Amunts K, Knoll A, Meier KH, Lippert T, Pennartz C, Ryvlin P, Destexhe A, Jirsa V, D’Angelo E, Bjaalie J. (2019) The Human Brain Project – synergy between neuroscience, computing, informatics and brain-inspired technologies. PLoS Biology 17: e3000344. https://doi.org/10.1371/journal.pbio.3000344 .
Pennartz CMA (2018) Consciousness, representation, action: the importance of being goal-directed. Trends in Cognitive Sciences 22:137-153.
Dora S, Pennartz CMA, Bohte S (2018) A deep predictive coding network for inferring hierarchical causes underlying sensory inputs. In: Kůrková V., Manolopoulos Y., Hammer B., Iliadis L., Maglogiannis I. (eds.) Artificial Neural Networks and Machine Learning – ICANN 2018. Lecture Notes in Computer Science, vol 11141. Springer, Cham. https://doi.org/10.1007/978-3-030-01424-7_45 .
Olcese U, Bos JJ, Vinck M, Van Mourik-Donga LB, Pennartz CMA (2018) Functional determinants of enhanced and depressed inter-areal information flow in NREM sleep between neuronal ensembles in rat cortex and hippocampus. Sleep 41: 1-18. https://doi.org/10.1093/sleep/zsy167 .
Stoianov I, Lansink CS, Pennartz CMA, Pezzulo G (2018) Model-based spatial navigation in the hippocampus-ventral striatum circuit: a computational analysis. Plos Comput. Biology 14 (9): e1006316. https://doi.org/10.1371/journal.pcbi.1006316 .
Meijer GT, Pie JL, Dolman TL, Pennartz CMA, Lansink CS (2018) Audiovisual integration enhances perceptual performance of stimulus detection. Frontiers in Behav. Neurosci. 12:231. https://doi.org/10.3389/fnbeh.2018.00231 .
Olcese U, oude Lohuis M, Pennartz CMA (2018) Sensory processing across conscious and nonconscious brain states: from single neurons to distributed networks for inferential representation. Front. Syst. Neuroscience 12:49. https://doi.org/10.3389/fnsys.2018.00049 (invited review).