Cognition and Language

PSY 511.003

Published

March 28, 2024

The emergence of complex behavior

Cambrian Explosion

Sparked by behavioral imperatives? (Fox, 2016)

Behavior requires energy
Behavior requires perception at a distance
Behavior requires action
Actions require
- Problem solving, (sequence) planning
- Current + stored information (memory)

Behaviors realized through…

Perception at a distance of what/where
Locomotion
- Approach/avoid/explore
Object manipulation/consumption
Signaling/communication
Physiological regulation

Complex behavior ~ Nervous systems

Cognition

Combines…

Perception
Attention
Imagery
Learning and conditioning
Memory
- Episodic (events)
- Semantic (facts, things, entities)
- Procedural (actions)
- Working
Problem-solving
- Planning, executing, evaluating sequences of behavior
Language

A form of computation?

https://www.explainthatstuff.com/howcomputerswork.html

In digital computers

Data are in binary (0,1) digit (bit) form
- Same sequence of bits could be a letter, a number, part of an image, a bit of sound, etc.
Operations on data, e.g., add, store, also in binary form
Distinct (physically separate) circuits for
- Input (keyboards, mice, cameras, mics, network cards)
- Processing (Central Processing Units or CPUs; Graphic Processing Units or GPUs)
- Storage (RAM, flash drives, hard drives, etc.)
- Output (displays, speakers, printers network commands)
Software
- Files that cause operations on data (e.g., implement algorithms)

https://www.geeksforgeeks.org/introduction-to-algorithms/

But…

There is a popular belief in neuroscience that we are primarily data limited, and that producing large, multimodal, and complex datasets will, with the help of advanced data analysis algorithms, lead to fundamental insights into the way the brain processes information. These datasets do not yet exist, and if they did we would have no way of evaluating whether or not the algorithmically-generated insights were sufficient or even correct. To address this, here we take a classical microprocessor as a model organism, and use our ability to perform arbitrary experiments on it to see if popular data analysis methods from neuroscience can elucidate the way it processes information.

…current analytic approaches in neuroscience may fall short of producing meaningful understanding of neural systems, regardless of the amount of data. (Jonas & Kording, 2017)

Cognition and the cerebral cortex

Cortical Macrostructure

Areas
- Unimodal sensory
- Polymodal association
- Motor
Connections
- Association
- Commissural

Cortical Microstructure

Columnar
Regional cytoarchitectonic differences

Wikipedia: Constantin von Economo’s cytoarchitectonic chart of the human brain - Laminar

Layer	Connection type	Comments
I		Few cell bodies
II	Efferent	Ipsilateral association via large pyramidal cells
III	Efferent	Contralateral commissural
IV	Afferent	from thalamus; small stellate & granual cells; V1 has sublayers
V	Efferent	Superficial -> Basal ganglia; Deep -> brainstem, spinal cord; pyramidal cells
VI	Efferent	Thalamus

Processing networks

“Although it has long been assumed that cognitive functions are attributable to the isolated operations of single brain areas, we demonstrate that the weight of evidence has now shifted in support of the view that cognition results from the dynamic interactions of distributed brain areas operating in large-scale networks….”

(Bressler & Menon, 2010)

Data-driven dynamics

Cortical states have high dimensionality
Is there a lower-dimensional space that maps onto behavior?

(Shine et al., 2019)

Data from \(n=200\) adult participants in Human Connectome Project (HCP)
7 cognitive tasks
Dimension reduction via principal components analysis (PCA)

Map PCAs to time series…

(Shine et al., 2019). Fig. 2: The low-dimensional signature across cognitive tasks. a, The procedure used to partition tPC1 into unique phases: low (blue), rise (red), high (orange), and fall (light blue). b, Scatter plot comparing the loading of tPC1 (colored according to the partition defined in a) with a temporal stability measure (defined by the similarity of the BOLD response at adjacent time points); we observed a significant positive Pearson’s correlation (r = 0.58) between |tPC1| and temporal stability (n=1,939 time points), providing heuristic evidence for attractor basins at the extremes of tPC1 engagement. c, A three-dimensional scatter plot comparing the first three tPCs; each node represents one time point (colored according to the phase of tPC1), with time implicitly unfolding across the embedding space (contiguous points connected by black line). d, The low-dimensional manifold traversed by the global brain state across the first three dimensions, with arrows depicting the direction of flow along the manifold.

How do these brain states map to cognition?
Explore overlap with NeuroSynth ‘topic families’

The results of our multimodal analysis revealed that the neural activity required for the execution of cognitive tasks corresponds to flow within a low-dimensional state space[43]. Across multiple, diverse cognitive tasks, the dynamics of large-scale brain activity engage an integrative core of brain regions that maximizes information-processing complexity and facilitates cognitive performance; only to then dissipate as the tasks conclude, flowing towards a more segregated architecture…Across multiple cognitive tasks with markedly different behavioral requirements, the dynamics of human brain activity were found to occupy a low-dimensional state space embedding that may form the functional backbone of cognition in the human brain.

(Shine et al., 2019)

The neural bases of cognitive control (Cai, Taghia, & Menon, 2024)

We discover a shared brain state across seven distinct cognitive tasks and found that the dynamics of this shared brain state predicted cognitive control abilities in each task.

(Cai et al., 2024)

(Figure 1 from Cai et al., 2024). We applied a Bayesian switching dynamic systems (BSDS) to time series extracted from brain regions constituting a multiple-demand system involved in cognitive control. We investigated latent brain states across seven different cognitive tasks and four different datasets. First, to demonstrate a shared latent brain state across cognitive domains, we matched task-specific latent brain states with the task-optimal latent brain state SHWM associated with high-load working memory task condition. Two different state matching algorithms demonstrated convergent results. Second, the relationship between the shared latent brain state and cognitive performance in each task was investigated using both multivariate canonical correlation and univariate correlation analysis. Third, we examined the shared latent brain state and its relation to clinical measures of inattention in a developmental cohort. Fourth, we evaluated similarity of latent brain states within individuals across cognitive tasks. WM n-back working memory task, AxCPT Ax continuous performance task, CuedTS Cued task switching task, Stern Sternberg working memory task, Stroop Stroop interference task, SST Stop-signal task, RP Relation processing task. The regression estimate is presented with 95% confidence interval (shaded area).

Data from Human Connectome Project (HCP) (\(n=90\) adults) and Dual Mechanisms of Cognitive Control (DMCC) project (\(n=50\) adults)
n-back reference task compared with other tasks: AxCPT, Cued task-switching, Sternberg memory task, Stroop.
What patterns of brain activation are similar in space and time to one another AND predict behavior (task performance)?
Additional goal was to compare these states to a sample of children with Attention Deficit Hyperactivity Disorder (ADHD)

(Figure 3 from Cai et al., 2024). Multivariate CCA revealed significant correlations between occupancy rates of latent brain states and behavioral variables in all the DMCC tasks, including (a) AxCPT, (b) CuedTS, (c) Sternberg and (d) Stroop (N=50). In each task, the component in which linear combination of behavioral variables that best represents general cognitive control was selected to investigate the relationship between latent brain state and behavioral performance. Weights of canonical components in each task are presented next to the axes and summarized in Supplementary Table S3. Univariate Pearson’s correlation revealed significant correlation between occupancy rate of the multiple-demand brain state (e.g., SHAxCPT) and cognitive control index in all the DMCC tasks, including (e) AxCPT, (f) CuedTS, (g) Sternberg task, and (h) Stroop. SHAxCPT, SHCuedTS, SHStern and SHStroop refers to the dynamic brain state that matches to SHWM in the AxCPT, CuedTS, Sternberg and Stroop tasks, respectively. AxCPT Ax Continuous performance task, CuedTS Cued task switching task, ACC Accuracy, RT Reaction time, LL Low load, HL High load, Con Congruent; Incon Incongruent, OR Occupancy rate. The regression estimate is presented with 95% confidence interval (shaded area). Source data are provided as a Source data file. P values were not adjusted for multiple comparisons.

Summing up

Cognition involves
- Do what, where, when, and how
The “cognitive” cortex
- Dorsolateral prefrontal cortex (DLPFC)
- Bilateral intraparietal sulcus
- Bilateral anterior insula
- Bilateral middle frontal gyrus (MFG)
- Posterior cingulate cortex (PG)
- Ventromedial prefrontal cortex (VMPFC)
Processing networks
- Functional specialization
- Dynamic interaction
- Low dimensional dynamics
- Nested feedback control loops

What do we want to know?
- What parts of the nervous system are evoked by cognitive process X? (localization)
- How does neural data support/undermine theory X of cognition?
- “…our survey nevertheless still makes it clear that very few resources are currently being devoted to using neuroimaging data to test theories about cognition.” (Tressoldi, Sella, Coltheart, & Umiltà, 2012)
- Also (Coltheart, 2013)
Neuroscience can constrain models of cognition (White & Poldrack, 2013)
- One process or two
- Serial vs. parallel processing
Show me your (cognitive) model…

(Figure 1 from Anderson, Pyke, & Fincham, 2016). Fig. 1. Illustration showing the durations of the four stages associated with problem solving. In the four example problems, the arrows denote new mathematical operators that participants had learned. In each stage, the axial slice (x = 0 mm, y = 0 mm, z = 28 mm in Talairach space) highlights brain regions in which activation in that stage was significantly greater than the average activation during problem solving. Brain images are displayed with the left hemisphere on the right-hand side.

(Figure 4 from Anderson et al., 2016). Fig. 4. The four brain signatures placed in a 3-D space where the activity of a stage is a sum of the activity of the signature in the solving stage plus a sum of the three vectors weighted by their coordinates in the space. The heat maps illustrate the proportion of change in activation relative to baseline. The coordinates of the stages are as follows (in Talairach space)—encoding: x = 1.61, y = 0.37, z = 0.58; planning: x = 0.58, y = 0.28, z = 1.38; solving: x = 0, y = 0, z = 0; and responding: x = 0.37, y = 1.78, z = 0.28. Brain images are displayed with the left hemisphere on the right-hand side.

Language and the brain

Language behavior

Productive
- Speaking (2-5 words/s), modulate prosody (intonation), often combined with gesture
- Writing, typing (.5-1.5 words/s)
Receptive
- Listening, responding (facial expressions, gestures, laughter, etc.)
- Reading (3-5 words/s)
How so fast? Time for feedback?

Hierarchical structure of language information

Phonetic
- |Ber| |wiTH| |mē|
Syntactic
Semantic

Pragmatic

Wernicke-Geschwind (WG) model

Carl Wernicke
Norman Geschwind
Perception ≠ production

Wernicke’s area (Brodmann Area or BA 42)

Adjacent to primary auditory cortex (A1; Heschl’s gyrus; BA 41)
Perception
Receptive or ‘fluent’ aphasia

Broca’s area

Inferior frontal gyrus, pars opercularis (BA 44) & pars angularis (BA 45)
Production
Expressive aphasia

Dual streams (Hickok & Poeppel, 2007)?

Ventral (speech signals -> semantics)
Dorsal (speech signal acoustics -> articulatory networks in frontal lobe)

Metaanalytic evidence (Hagoort & Indefrey, 2014)

(Hagoort & Indefrey, 2014). Figure 1. Schematic representation of the brain showing regions with reliably reported activations for sentences compared with nonsentential stimuli (a) and sentences with high syntactic or semantic processing demands compared with simpler sentences (b,c). The left posterior inferior frontal gyrus is further subdivided into Brodmann areas (BA) 44 (above black line), BA 45 (below black line, above AC–PC line) and BA 47 (below AC–PC line). Green regions indicate a reliable number of reports. Pink regions indicate no reports in 53 studies. For details, see Supplemental Tables 2, 3, and 4. Abbreviations: AC, anterior commissure; PC, posterior commissure).

(Hagoort & Indefrey, 2014). Figure 2. (a) Summary of activation patterns for sentences with high syntactic or semantic processing demands compared with simpler sentences. (b) Syntactic/semantic gradients in left inferior frontal and posterior temporal cortex based on 28 studies reporting posterior temporal cortex activation for syntactically demanding or semantically demanding sentences compared with less demanding sentences (see Supplemental Figure 13 for details). The centers represent the mean coordinates of the local maxima, and the radii represent the standard deviations of the distance between the local maxima and their means. Abbreviations: GFm, GFi, middle and inferior frontal gyri; BA, Brodmann area; GTs, GTm, GTi, superior, middle, and inferior temporal gyri; STS, ITS, superior and inferior temporal sulci; Gsm, supramarginal gyrus.

“A meta-analysis of numerous neuroimaging studies reveals a clear dorsal/ventral gradient in both left inferior frontal cortex and left posterior temporal cortex, with dorsal foci for syntactic processing and ventral foci for semantic processing. In addition…further networks need to be recruited to realize language-driven communication to its full extent.”

(Hagoort & Indefrey, 2014)

Summing up

WG model incomplete, simplistic
- Broca’s not just production; Wernicke’s not just perception
- Beyond single words…
Rapid, fluent comprehension and production of language relies on
- Distributed temporal/frontal networks
- Efficient bottom-up and top-down processing
- Syntactic vs. semantic/articulatory processing

References

Anderson, J. R., Pyke, A. A., & Fincham, J. M. (2016). Hidden stages of cognition revealed in patterns of brain activation. Psychological Science, 27(9), 1215–1226. https://doi.org/10.1177/0956797616654912

Bressler, S. L., & Menon, V. (2010). Large-scale brain networks in cognition: Emerging methods and principles. Trends in Cognitive Sciences, 14(6), 277–290. https://doi.org/10.1016/j.tics.2010.04.004

Cai, W., Taghia, J., & Menon, V. (2024). A multi-demand operating system underlying diverse cognitive tasks. Nature Communications, 15(1), 2185. https://doi.org/10.1038/s41467-024-46511-5

Coltheart, M. (2013). How can functional neuroimaging inform cognitive theories? Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 8(1), 98–103. https://doi.org/10.1177/1745691612469208

Fox, D. (2016). What sparked the Cambrian explosion? Nature, 530(7590), 268–270. https://doi.org/10.1038/530268a

Hagoort, P., & Indefrey, P. (2014). The neurobiology of language beyond single words. Annu. Rev. Neurosci., 37, 347–362. https://doi.org/10.1146/annurev-neuro-071013-013847

Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nat. Rev. Neurosci., 8(5), 393–402. https://doi.org/10.1038/nrn2113

Jonas, E., & Kording, K. P. (2017). Could a neuroscientist understand a microprocessor? PLoS Computational Biology, 13(1), e1005268. https://doi.org/10.1371/journal.pcbi.1005268

Powers, W. T. (1973). Behavior: The control of perception. Aldine Chicago. Retrieved from http://www.pctresources.com/Other/Reviews/BCP_book.pdf

Shine, J. M., Breakspear, M., Bell, P. T., Ehgoetz Martens, K. A., Shine, R., Koyejo, O., … Poldrack, R. A. (2019). Human cognition involves the dynamic integration of neural activity and neuromodulatory systems. Nature Neuroscience, 22(2), 289–296. https://doi.org/10.1038/s41593-018-0312-0

Swanson, L. W. (2005). Anatomy of the soul as reflected in the cerebral hemispheres: Neural circuits underlying voluntary control of basic motivated behaviors. Journal of Comparative Neurology, 493(1), 122–131. https://doi.org/10.1002/cne.20733

Swanson, L. W. (2012). Brain architecture: Understanding the basic plan. Oxford University Press.

Tressoldi, P. E., Sella, F., Coltheart, M., & Umiltà, C. (2012). Using functional neuroimaging to test theories of cognition: A selective survey of studies from 2007 to 2011 as a contribution to the decade of the mind initiative. Cortex; a Journal Devoted to the Study of the Nervous System and Behavior, 48(9), 1247–1250. https://doi.org/10.1016/j.cortex.2012.05.024

White, C. N., & Poldrack, R. A. (2013). Using fMRI to constrain theories of cognition. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 8(1), 79–83. https://doi.org/10.1177/1745691612469029