Arts and Cognition Monograph: Developing and Implementing Neuroimaging Tools

March, 2008

By Mark D’Esposito, M.D., University of California, Berkeley 


Over the last three years we have developed and used advanced functional magnetic resonance imaging (fMRI) tools that can address the question of whether arts training impacts the brain. We: 1) developed multivariate statistical analyses for fMRI data; 2) developed fMRI biomarkers of cognitive processes; 3) employed new fMRI imaging tools to investigate neural processes underlying music practice; and 4) undertook fMRI imaging in pianists and non-pianists to investigate the neural processes underlying long-term expertise gained through “slow learning” compared to “fast learning.” Slow learning is characterized by gradual performance improvements that produce structural and functional changes in the brain. Fast learning involves rapid improvements in performance that lead to habituation-like brain changes, perhaps due to changes in synaptic strength.

Since we contend that current fMRI data analytic methods have been inadequate for fully assessing the types of changes that may occur in the brain with learning, we developed numerous new methods. These methods combine univariate and multivariate approaches for analyzing fMRI data, to assess the impact of formal arts training on the brain. We have used these tools and made them available to consortium members for their use in analyzing their own fMRI datasets.

We gathered data to test the Consortium’s hypothesis that cognitive processes that are facilitated by training in the arts are transferred successfully to other cognitive domains. We explored whether fMRI measurements can effectively serve as a biomarker—an indicator—signaling an event or condition that gives a measure of exposure, effect, or susceptibility. With a reliable biomarker, we can quantify the effect of arts training on the developing brain of a young person. Our research focused on the system of “cognitive control,” thought to be a critical function of the prefrontal cortex. This system allows us to flexibly guide our behavior, and is critical for all types of learning.

We derived several fMRI biomarkers by using a task that directly assesses the neural mechanisms underlying top-down (cognitive control) changes. We investigated the cognitive control processes involved when 60 study participants were imaged as they performed two memory tasks. The tasks required that they enhance relevant, and suppress irrelevant, information. The fMRI measurements are derived from both univariate as well as multivariate data analyses. We also asked the participants to fill out musical training questionnaires. Based on data from 40 of the 60 participants analyzed to date, we categorized participants by whether or not they could read music, and by whether or not they had received formal music training. We assessed differences between these groups on our fMRI biomarkers.

We found that participants with formal musical training showed significantly stronger neural enhancement and suppression effects, indicating better cognitive control. Moreover, since the memory tasks had no specific linkage to reading music, these results suggest that formal musical training may generalize, by having an impact on other brain systems that are different than those affected by music training.

Our empirical studies of task practice and motor learning in pianists have laid a foundation for understanding the neural mechanisms by which formal arts training may impact the brain. The next step will be to develop interventions that can provide evidence that formal arts training caused the observed changes in cognition and brain function that have been demonstrated in correlative studies. Reliable, quantifiable fMRI biomarkers will be necessary to assess such causal effects on the brain.


Theories of brain organization focus on two distinct, but complementary principles: modularity, the existence of neuronal assemblies with intrinsic functional specialization; and network connectivity, the integration of information from distributed brain regions resulting in organized behavior. While the modular model may be reasonable to describe fundamental features of the function of primary cortices (e.g., primary motor or visual cortex), it is insufficient to explain complex cognitive processes that cannot be localized to isolated brain regions. Rather, cognitive abilities emerge from contextual relations that are subserved by widespread cortical connections.

Functional MRI (fMRI) in humans is ideally and uniquely suited to explore neural networks, since it simultaneously records correlates of neural activity throughout the entire functioning brain with high spatial resolution. However, most fMRI studies utilize univariate analyses, permitting only the independent assessment of activity within each brain region in isolation of all others. With Dana Foundation funding, we have developed several multivariate approaches to analyze neuroimaging data in a manner that more directly addresses the network model of cognition. Also, if any fMRI measurement is expected to assess the effects of an intervention, such as training in the arts, that measure must be quantifiable and reliable. A second approach we have taken is to test whether fMRI data can provide a biomarker for changes in brain function.

In addition to developing novel functional neuroimaging tools, we are using these tools to investigate fundamental neural mechanisms that may mediate how formal arts training can impact brain development and function. We have focused on the neural mechanisms underlying our ability to adapt flexibly to new experiences with practice, as well as on mechanisms underlying fast learning (i.e., rapid improvements in performance that lead to automatization and habituation-like brain changes, perhaps due to changes in synaptic strength) versus slow learning (i.e., gradual performance improvements that cause functional reorganization and morphological changes to the brain).

Specific Research Approaches:

  1. Development of multivariate statistical analyses for functional MRI data.
  2. Development of fMRI biomarkers of cognitive processes.
  3. Using neuroimaging, investigation of the neural mechanisms underlying paractice (e.g., systematic training by multiple repetitions).
  4. Using neuroimaging, investigation of the neural mechanisms underlying long-term expertise (slow learning) versus fast learning, by studying pianists and nonpianists.

Specific Approach #1: Functional MRI multivariate analyses

Multivariate analyses of imaging data allow the generation of functional and effective connectivity maps of brain regions that interact within the framework of a distributed system to underlie emergent cognitive processes. These methods do not trivialize the functional specialization of brain regions, but rather emphasize the role of brain regions within the context of other covarying, anatomically connected, active brain regions, as well as the specific cognitive process that is being engaged. Over the past three years of funding, we have tested and validated four methods for use with fMRI data:


Coherence is a spectral measure that has been used in electroencephalography (EEG) and other imaging modalities to study functional relationships between different brain areas. Just prior to Dana funding, my lab demonstrated that coherence could also be used to investigate functional connectivity in fMRI data (Sun et al., 2004). Because coherence is invariant to inter-regional hemodynamic response differences, coherence may provide a more appropriate method for measuring functional connectivity than correlation or covariance measures.

... different cortical areas maintain relatively different types of information ... With Dana Foundation funding, we extended this method to include temporal measurements. That is, using coherence analyses, we can measure relative latencies between functionally connected brain regions using the phase-delay of fMRI data. Subsequently, using this method, we published two empirical studies (Curtis et al., 2005; Miller et al., 2005), demonstrating that this method can be applied to address the types of questions proposed and implemented by researchers in the consortium. For example, we previously demonstrated that different cortical areas maintain relatively different types of information when individuals are remembering that information across short periods of time (Curtis et al., 2004). Despite these differences in regional brain activity, we could only assume but not address the functional interactions between the identified nodes of the putative brain network. Thus, we used coherence to formally characterize functional interactions between these brain areas.

We found that the type of representational codes that are being maintained in working memory bias frontal-parietal interactions. For example, coherence between frontal eye fields (FEF) and other oculomotor areas were greater when a motor representation was an efficient strategy for remembering information across a delay period. However, coherence between the FEF and higher-order multimodal brain regions, e.g., prefrontal cortex, was greater when a sensory representation (e.g., the location of an object in space) must be maintained in working memory. We were also able to demonstrate that the timing of the events mediated by these brain regions differed during different stages of processing, such as the encoding and retrieval of memories. This type of temporal information, derived from these coherence analyses, cannot be obtained in traditional univariate analyses. Thus, it can provide valuable information about the sequence of cognitive processes within a brain network.

Granger Causality

We have also tested and validated a second multivariate method, called Granger causality, to analyze fMRI data (Kayser & D’Esposito, in revision). Granger causality is an exploratory multivariate method that allows one to make quantitative statements about the ability of one time series to predict another. In practice, this ability to make predictions (hence the term Granger “causality”) lies in the way in which time series are modeled. Imagine two time series, taken from two different voxels in the brain. Initially, the first of those time series is fit by a simple model that attempts to predict subsequent time points in the time series, based on previous time points. Because the fit of the model will not be perfect, there remains a residual, with a variance “Var1,” representing the portion of the time series for which the model does not account. If previous time points from the second time series are then also incorporated into the simple model of time series 1, the new fit results in a new residual, with variance “Var2.” If Var2 is less than Var1, then time series 2 is said to be “Granger causal” for time series 1, because it explains additional variance.

This technique has recently been adapted to fMRI data by Goebel and colleagues, and we extended its use by relying on its use in the econometrics literature (e.g., the so-called “conditional” Granger causality). Importantly, we compared it directly to coherence, in order to provide more information about multivariate methods in general, but also to determine how much it complemented, versus replicated, this method. Like coherence, Granger causality analyses further informed our understanding of this interregional connectivity. Importantly, in specific instances, Granger causality provided new information that coherence did not. Thus, Granger causality can also serve as a principled and integrated method of data analysis within the increasing array of multivariate techniques.

Information Theory

A third approach for analysis of event related fMRI data that can also assess functional connectivity is based on measures from information theory and is used both for spatial localization of task-related activity, as well as for extracting temporal information regarding the task dependent propagation of activation across different brain regions (Fuhrman et al., 2007). This approach enables whole brain visualization of areas most involved in coding of a specific task condition, the time at which they are most informative about the condition, as well as their average amplitude at that preferred time. An advantage of this approach is that it does not require prior assumptions about the shape of the hemodynamic response function, nor about linear relations between the fMRI BOLD signal and presented stimuli (or task conditions). We have demonstrated that relative delays between different brain regions could also be computed without prior knowledge of the experimental design, suggesting a general method that could be applied for analysis of differential time delays that occur during natural, uncontrolled conditions.

To validate this method, we analyzed fMRI data during performance of a motor learning task. We showed that during motor learning, the unimodal motor cortical activity preceded the response in higher-order multimodal association areas, including posterior parietal cortex. Brain areas found to be associated with reduced activity during motor learning, predominantly in prefrontal brain regions, were informative about the task typically at significantly later times. Importantly, these findings replicated our results using coherence and Granger causality (Kayser & D’Esposito, in revision).


Finally, just prior to Dana funding, we had developed a fourth approach for characterizing inter-regional interactions using event-related fMRI data (Rissman et al., 2004). This method’s principle advantage over existing analytical techniques is its ability to model the functional connectivity between brain regions during distinct stages of a cognitive task. The method is implemented by using separate covariates to model the activity evoked during each stage of each individual trial in the context of the general linear model (GLM). The resulting parameter estimates (beta values) are sorted according to the stage from which they were derived, to form a set of stage-specific beta series. Regions with beta series that are correlated during a given stage are inferred to be functionally interacting during that stage.

We published an empirical paper demonstrating the utility of this method during the performance of cognitive tasks (Gazzaley et al., 2005). In a working memory task, a beta-correlation analysis revealed a network of brain regions exhibiting significant correlations with the prefrontal cortex during the working memory delay period, even though there was minimal activity in this network of brain regions when each region was analyzed independently using univariate analyses. Again, these findings support the notion that the coordinated functional interaction between nodes of a widely distributed brain network underlies cognitive processing that cannot be revealed in traditional analyses.

Application of our fMRI Methodology to the Questions Funded by the Dana Grant

Over the past three years, we have collected our own empirical fMRI data during the performance of several cognitive paradigms that not only will be useful for continuing to validate our multivariate analytic methods, but also will provide data that can directly address the hypotheses set forth by the Dana Foundation consortium (see below under Specific Approach #3 and #4). Also, we have made our fMRI data analytic tools available to all of the researchers in the Dana Foundation consortium. Sharing such analytical tools is quite labor intensive, since each functional imaging laboratory collects data from different MRI scanners, and uses different computer workstations, platforms and software to analyze the fMRI data it collects.

Also each of our fMRI data methods is computationally intensive and requires sophisticated algorithms to implement. We have written software for these tools in Matlab, which is widely available to most imaging laboratories, although we must write extensive documentation for these programs in order for them to be implemented by students, post-doctoral fellows, and faculty in other labs. This software, and set-up on-site training, is available to all consortium investigators who have collected fMRI data, so that they can extract additional information from their data by adding multivariate approaches to the univariate approach they have already undertaken.

Such new directions in fMRI data analysis should provide further insight and a more complete understanding of the neural mechanisms underlying the influence of formal arts training on the developing brain. These tools also have been provided to non-consortium laboratories from around the world, already resulting in several published papers.

Specific Approach #2: Functional MRI biomarkers

Any study of the “impact of the arts on the brain,” in my opinion, must have a reliable marker of “brain” function that can be measured and quantified. “Patterns of brain activity” which is the most commonly reported fMRI measurement cannot be easily quantified and tested for its reliability. Thus, how can we use fMRI to measure whether formal arts’ training has had an impact on the brain without the current existence of such markers? Functional MRI definitely has the potential to provide the types of measurements we need to test our hypotheses.

... cognitive control is a system that is critical for all types of learning. However, the most commonly used methods for analyzing fMRI data thus far have not tapped into the full potential of this data. For example, most fMRI studies of individuals who have had formal arts training (e.g., musician, dancers), typically report only patterns of brain activity (or brain maps) that differentiate them from groups of individuals that have not had such training. These analyses do not produce reliable, quantifiable measurements that reflect the underlying neural systems that are being studied. And, in most instances, such fMRI findings only identify “where” changes due to training are occurring in the brain, without providing any insight into the neural mechanisms mediating those changes.

A biomarker is an indicator, signaling an event or condition in a biological system or sample, and giving a measure of exposure, effect, or susceptibility. Can fMRI measurements serve as biomarkers? For several years, with support from the Dana Foundation, my lab has been trying to answer this question. The premise that we are working on is that if we have a reliable biomarker of the neural system we wish to study, we can reliably quantify how such a neural system is affected by almost any input. The input may be the effects of a drug, the effects of cognitive therapy, it may be the effects of a disease process, or it may be the effects of formal arts training on the developing brain of a young individual.

The neural system that my lab investigates is the one that underlies cognitive control. Cognitive control allows us to flexibly guide our behavior. Goal–directed behavior is clearly guided by an interaction of top-down and bottom-up processes. By bottom-up, I mean those processes that guide automatic behavior and are determined by the nature of sensory input. By top-down, I mean those processes that guide behavior that is determined by internal states such as knowledge from previous experience, expectations, and goals. Without cognitive control we would be unable to overcome reflexively triggered instinctive behaviors that are impervious to the context of the situation. Thus, cognitive control is a system that is critical for all types of learning. It is likely that the neural system mediating cognitive control is influenced by formal arts training. In addition, the integrity of this system may be a critical determining factor regarding how formal arts training impacts brain function.

We have derived several fMRI biomarkers by using a task that directly assesses the neural mechanisms underlying top-down modulation by investigating the processes involved when study participants are required to enhance relevant, and suppress irrelevant, information. During each trial of the task, participants observe sequences of two faces and two natural scenes presented in a randomized order.

The tasks differed in the instructions informing the participants on how to process the stimuli: 1) Remember Faces and Ignore Scenes, 2) Remember Scenes and Ignore Faces, or 3) Passively View faces and scenes without attempting to remember them. In each task, the period in which the cue stimuli were presented was balanced for bottom-up visual information, thus allowing us to probe the influence of goal-directed behavior on neural activity (top-down modulation). In the two memory tasks, the encoding of the task-relevant stimuli requires selective attention and thus permits the dissociation of physiological measures that index the enhancement of relevant information, versus the suppression of irrelevant information. These measurements are derived from both univariate as well as multivariate analyses.

A causative study is the next logical direction ... We gave musical training questionnaires to all participants who have performed this cognitive control task during fMRI scanning. We currently have data from about 60 participants, although we have analyzed data from about 40 of these. In our first analysis of this data, we simply categorized subjects as those who could read music or not, and those who had formal music training or not. We have assessed differences between these groups of participants on our fMRI biomarkers. We found that participants who had formal musical training showed show significantly stronger neural enhancement and suppression effects (as measured by fMRI), indicating better cognitive control. And, these participants performed better on this cognitive control task. Importantly, unlike other studies that typically test participants with fMRI on tasks that are similar to types of expertise they have, this task has no specific linkage to reading music. These results suggest that formal musical training may generalize by having an impact on other brain systems that are different than those affected by training.

Alternatively, our participants with formal musical training may have had a “better” cognitive control system prior to formal musical training. These two possible explanations for our fMRI highlight the difference between a “correlative study,” which this was, and a “causative” study, which can distinguish between these alternative explanations. A causative study is the next logical direction for this line of investigation. However, this fMRI study does highlight an approach in which one can study the impact of formal arts training on neural systems that differ from those neural systems that were trained. In this particular study, there are numerous additional analyses that can be performed on this rich dataset, which will be performed in our laboratory in the future.

Specific Approach #3: Neural mechanisms underlying practice

Previous functional neuroimaging studies have shown that neural activity changes with task practice. The types of changes reported have been inconsistent, however, and the neural mechanisms involved remain unclear. In an fMRI study (Landau et al., 2004), we investigated the influence of practice on different component processes of working memory (WM), on a similar paradigm as described above. Event-related fMRI allowed us to examine signal changes from early to late in the scanning session (lasting approximately 1 hour) within different task stages (i.e., encoding, delay, retrieval). Event-related fMRI also enabled us to determine the influence of different levels of WM load on neural activity. We found practice-related decreases in fMRI signal and effects of memory load occurring primarily during memory encoding. This suggests that practice improves the efficiency of memory encoding, especially at higher memory loads, through a mechanism of neural efficiency.

The fMRI signal decreases we observed were not accompanied by improved behavioral performance, indicating that practice-related changes in activation may occur during a scanning session without behavioral evidence of learning. Our results challenge the idea that dynamic changes in activation are linked to faster or more accurate performance, as has been commonly reported in experiments on cognitive and motor skill learning. Instead, the neural activity we observed changes over time, but it is independent of task improvement, suggesting that there are important neural changes associated with learning that are not captured in the behavioral data.

In a follow-up fMRI study (Landau et al., in press), we extended this work to investigate how practice on cognitive tasks affects different types of cortical regions (e.g., highly specialized primary sensory cortex compared to unimodal, compared to multimodal associative cortex). Little is known about whether task practice influences these types of regions differently. We used event-related fMRI to examine practice-related activation changes in different region types over the course of a scanning session while participants performed a WM task. We observed significant decreases, and not increases, in fMRI signal that occurred primarily during WM encoding in multimodal and unimodal regions, but not in primary sensory regions.

Furthermore, multimodal lateral frontal regions decreased by 39.4% during the cue, which was disproportionately greater than the 8.1% decrease for primary regions. These findings indicate that task practice does not have a uniform effect on stages of cognitive processing or on different brain regions. Instead, regions engaged during specific stages of processing (such as encoding or retrieval), may have greater capacity for functional plasticity than other processing stages. Additionally, the degree of specialization within brain regions may determine their processing efficiency. Multimodal and unimodal regions may be specialized for flexible experience-related change, while those supporting primary sensory processing may operate at optimal efficiency and are less susceptible to practice.

Fast learning refers to rapid improvements in performance, leading to automatization and habituation-like brain changes ... Together, these two fMRI studies have provided valuable insight into the neural mechanisms underlying task practice. Such mechanisms are likely fundamental to all types of learning, and suggest that different cognitive processes and cortical regions are affected by practice in different ways. The next logical step is to examine these practice-related changes during more extended practice, such as that experienced during formal arts training, as well as to compare these changes in individuals who have already undergone formal training compared to those who have not.

Specific Approach #4: Neural mechanisms underlying slow and fast learning

Previous studies of motor learning have proposed a distinction between “fast” and “slow” learning. Fast learning refers to rapid improvements in performance, leading to automatization and habituation-like brain changes, perhaps due to changes in synaptic strength. Slow learning refers to gradual performance improvements causing functional reorganization and morphological changes to the brain. These mechanisms have rarely been examined simultaneously, which was the focus of another fMRI study we performed (Landau et al., 2006).

We examined the influence of long-term motor expertise (“slow” learning) while pianists and non-pianists performed alternating epochs of sequenced and random stimuli (“fast” learning) during functional MRI scanning. All study participants demonstrated learning of the sequence, as demonstrated by decreasing reaction times (RTs) on sequence trials relative to random trials, throughout the session. Pianists also demonstrated faster RTs and superior sequence acquisition compared with non-pianists. Within-session decreases in primary sensorimotor cortex and multimodal parietal cortex was observed in both groups.

Overall, the results of this experiment support the hypothesis ... "training in the Arts changes the brain." Additionally, there was more extensive activation throughout the session for pianists compared with non-pianists across a distributed prefrontal-parietal network. These findings provide evidence that different neural systems subserve “slow” versus “fast” phases of learning. Importantly, pianists, who have undergone long-term motor training, recruited an enlarged brain network that included both motor and nonmotor regions. This suggests that slow learning mechanisms selectively modify the efficiency of regions specific to the domain of expertise, as well as that of higher-level associative regions.

Our next set of analyses with this data is to apply our different multivariate methods to investigate changes in functional connectivity between pianists and non-pianists to gain further insight into the neural mechanisms underlying fast vs. slow learning. Overall, the results of this experiment support the first general hypothesis of the Dana Foundation grant, that is, “training in the arts changes the brain.”

Concluding Comments

Our three years of research have led to the development of numerous novel, advanced methods for analyzing fMRI data to assess brain function. It is our opinion that both univariate, and multivariate approaches towards fMRI data must be utilized to fully understand the neural mechanisms underlying the impact of formal arts training on the brain. In our research related to the hypotheses of the Dana Foundation grant, we have utilized these tools. Moreover, we have made these tools available to Dana Foundation consortium members in order to analyze their own fMRI datasets. Finally, our empirical studies of task practice, as well as motor learning in pianists, have laid a foundation for understanding the neural mechanisms by which formal arts training may impact the brain.

In my opinion, the next step will be to develop interventions that can provide evidence that formal arts training caused the observed changes in cognition and brain function that have been demonstrated in correlative studies. Reliable, quantifiable fMRI biomarkers will be necessary to assess such causal effects on the brain. Unfortunately, little effort has been put forth by the users of neuroimaging to develop such biomarkers. Thus, it should be a continued priority of funding agencies to provide a balance of support between both empirical and methodological research.

back to top

Papers Supported by the Dana Grant

Sun, F.T., Miller, L.M., D’Esposito, M. Measuring temporal dynamics of functional networks using phase spectrum of fMRI data. NeuroImage, 28:227-37, 2005.

Curtis, C.E., Sun, F.T., Miller, L.M., D’Esposito, M. Coherence between fMRI time-series distinguishes two spatial working memory networks, NeuroImage, 26:177-83, 2005.

Miller, L.M., Sun, F.T., Curtis, C.E., D’Esposito, M. Functional interactions supporting inhibitory control during an oculomotor task, Human Brain Mapping, 26:119-27, 2005.

Landau, S.M., D’Esposito, M., Sequence learning in pianists and non-pianists: an fMRI study of motor expertise, Cognitive, Affective & Behavioral Neuroscience, 6:246-59, 2006.

Sun, F.T., Miller, L.M., Rao, A.A., D’Esposito, M. Functional connectivity during bimanual learning, Cerebral Cortex, 17:1227-34, 2007.

Fuhrmann-Alpert, G., Handwerker, D., Sun, F.T., D’Esposito, M., Knight, R.T. Information Analysis of Event-Related BOLD Responses: Exploring Spatio-temporal Patterns of Brain Activations, NeuroImage, 34:1545-1561, 2007.

Gazzaley, A., Rissman, J., Cooney, J., Rutman, A., Seibert, T., Clapp, W., D’Esposito, M. Functional interactions between prefrontal and visual association cortex contribute to top-down modulation of visual processing, Cerebral Cortex, in press.

Kayser, A.S., Sun, F.T., D’Esposito, M. A comparison of Granger causality with other multivariate techniques in the analysis of functional MRI data, Human Brain Mapping, in revision

Landau, S.M., Garavan, H., Schumacher, E.H., D’Esposito, M. Regional-specificity and practice: dynamic changes in object and spatial working memory, in press.

back to top