Tuesday, April 01, 2003

Economics Takes A Run at Brain Science’s Toughest Problems

Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics

By: Gualtiero Piccinini Ph.D.

rev_v5n2piccinini_3

Neuroscience has been moving from triumph to triumph, discovering how our brains generate the dazzling spectrum of human behavior. But mental functions such as how we choose between competing desires or how we predict an event remain hard to explain at the level of connecting a specific function with a specific neuronal network.

In this quest, Positron Emission Tomography (PET), Magnetic Resonance Imaging (MRI), and other methods for recording what happens in the brain have brought spectacular results. We must not only collect data, however, but apply the right concepts to interpreting them. Roughly speaking, understanding neural mechanisms means figuring out the relevance of each piece of the puzzle: what each component and activity contributes to the mechanism in question, what everything’s function is. Which proteins are controlled by which genes? Which signals are carried by which chemicals? What sensory information is processed by which sensory areas?

Scientists can observe what sensory receptors certain neurons are responding to, or what muscular movements are elicited by what neurons. For instance, neurons that are affected by heat receptors in the left hand apparently have the function of varying their excitation level based on the temperature encountered by that hand. Neurons whose signals lead to contractions of fibers in the left biceps apparently have the function of contracting the left biceps. This functional analysis is relatively straightforward for sensory and motor neurons in the peripheral (as contrasted with the central) nervous system and spine. Studying sensory and motor areas of the brain is harder because responses by cortical neurons are not strictly linked to specific sensory receptors or muscle fibers. To understand functions in the brain, neurophysiologists have had to enlarge their notion of a neuronal response. In the case of sensory areas, they try to identify what stimuli in the environment the neurons are responding to, regardless of what part of the skin (or retina, or whatever sensitive surface) is hit by the stimuli. In the case of motor areas, they try to identify what behaviors are elicited by neurons, regardless of which muscles are used for that behavior. This is tough, but at least the activity of neurons in sensory and motor areas of the cortex can be correlated with something observable— namely, stimuli and responses.

But what about functional analysis of neurons whose activity does not correlate with either stimuli or responses? For example, if a neuron’s activity is involved in predicting a certain event in the future, you cannot observe the correlation between the neural activity and the future event. The event the organism is predicting has not happened yet; it may never happen if the prediction is false. Or consider neurons involved in assessing which of two possible behaviors is more desirable. Even correlating the behaviors with the neural activity (if there were indeed a correlation) would not reveal whether the neural activity meant the organism deemed the behavior desirable or undesirable.

Glimcher proposes to augment the array of concepts and mathematical tools at the disposal of neuroscientists with three workhorses of economic science: probability theory, decision theory, and game theory.

These obstacles have recently spurred a new approach to non-sensory, non-motor brain areas, one that borrows key concepts from economics in order to get at mental processes (such as predicting the future or making value choices) that we would like to understand in terms of brain activity. Paul W. Glimcher, Ph.D., associate professor of neural science and psychology at New York University, is a pioneer in this field, and his new book, Decisions, Uncertainty, and the Brain, ushers in a field called neuro- economics. Glimcher proposes to augment the array of concepts and mathematical tools at the disposal of neuroscientists with three workhorses of economic science: probability theory, decision theory, and game theory.

THREE USEFUL THEORIES

These three theories are formal tools for economists, scientists, and statisticians, but they operate—albeit usually unwittingly— at the level of our everyday behavior. When you have only partial knowledge of the future, probability theory can help guide your thinking. Given a series of possible outcomes (say, of flipping a coin), probability theory tells you how to assign probabilities to them (heads or tails) in a coherent way: say, 50-50 or 40-60 but never 40-40 or 60-60. The theory also tells you how to update these predictions based on your observations. For example, if you start with a 60-40 assignment but then observe an equal number of heads and tails over many trials, probability theory tells you to move closer to a 50-50 assignment.

Further help when you are pondering an uncertain future is offered by decision theory. You are driving along the highway, for example, and want to stop to eat. You see an exit, but no restaurant signs. You think you might find some food anyway, but it is unlikely to be good food. You also think there might be another exit soon, with a good restaurant for sure. Depending on what probabilities you assign to finding food under those two conditions (take the first exit or wait), and depending on how much you care about good food, decision theory tells you which is better from the point of view of your desire to eat well. Decision theory tells you, when you face different possible outcomes, each with a certain value for you, which one you should choose if you want to maximize your expected satisfaction—what economists call the utility of the decision.

Finally, suppose you face an opponent and your expected utility—the desirability of a particular outcome for you—depends on what your opponent does. If you attack him and he flees, you win everything he has. But if he fights back, you may or may not win anything, and you are likely to get hurt. Therefore, in deciding whether to attack, you should consider what he is likely to do. Game theory tells you what you should do in the presence of opponents who are also wondering what to do. If you follow game theory, you will not always win, but, under specified conditions, you will do as well as it is possible to do.

Decision theory and game theory (both of which make use of probability theory) are basic tools used to study human economic behavior. Biologists have also applied them successfully to the study of animal behavior. They assume animals have information about which events are likely to occur and what values those events have for what they call the animals’ inclusive fitness. (This refers to the rate at which an animal’s genes are propagated.) Biologists have thereby shown that many animals behave to a certain degree as if they were attempting to maximize their inclusive fitness in the way predicted by decision theory and game theory. In Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics, Paul Glimcher takes all these ideas to the next step, suggesting that there are identifiable neural mechanisms underlying those behaviors in animals and people.

According to decision theory, an animal facing an uncertain future can maximize its expected utility (passing along its genetic inheritance) if it can in some way take into account two variables: the probability of a given outcome and its value. Glimcher and a colleague, Michael Platt, boldly searched monkeys’ brains for activity that correlated with those variables. 

NEUROECONOMICS IN MONKEYS

According to decision theory, an animal facing an uncertain future can maximize its expected utility (passing along its genetic inheritance) if it can in some way take into account two variables: the probability of a given outcome and its value. Glimcher and a colleague, Michael Platt, boldly searched monkeys’ brains for activity that correlated with those variables. In a small portion of the parietal cortex (the lateral intraparietal area, or LIP), they believe they found it.

How does one know whether a monkey is responding to probabilities? In designing their research, Glimcher and Platt varied the frequency with which the same behavior by the same monkey yielded a reward (for example, a squirt of fruit juice). Frequency, of course, is one way that probability manifests itself. Thus, based on the frequency of reward, a monkey could in principle estimate the reward’s probability. Since choosing what to do to get a reward requires keeping track of the probability of that reward, it follows that some activity of the monkey’s neurons should vary along with the likelihood that a chosen action would yield a reward. Glimcher and Platt found neurons doing exactly that. Particular neurons in the LIP responded strongly when there was a high probability of reward and weakly when there was a low probability. Moreover, the level of neuronal activity was adjusted in response to stimuli that raised or lowered the probability of the event, just as probability theory suggests that it would. That is, the level of activity of these neurons varied based not just on the past frequency of reward, but also on the instantaneous probability of reward as could be predicted from currently available evidence.

How does this logic apply to finding possible neuronal correlates of processing the value of an outcome? Glimcher and Platt kept the sensory and motor properties of the task unchanged, while changing the amount of reward that a monkey would get. For a while, a certain movement would yield a fixed amount of fruit juice, so that the monkey could learn to expect that reward. Then Glimcher and Platt increased or decreased the amount of juice received for that same movement, until the monkey learned to expect that new reward. They found neurons in the LIP that fired more strongly when the animal expected to receive a large, rather than a small, reward. This suggests that the function of these neurons may be to vary along with the expected value of an action.

Glimcher and Platt also tested whether these neurons influenced what the monkey actually did. They ran an experiment in which the monkey could choose how to move while they changed the reward— hence the value—coming from the possible movements. They showed that both the probability that the monkey would select a movement and the activity of the relevant neurons were correlated with the value of the movement. This is exactly what functional analysis in the brain is about.

As a result of the experiments, Glimcher tentatively concludes that neurons in the LIP influenced the monkey’s behavior in ways consistent with decision theory. He has not yet applied game theory to the same extent, but some observations of behavior and neurophysiology suggest its relevance. If Glimcher is right, decision theory and game theory can be exploited to give a rigorous working definition of new neurophysiologic variables—neural activity that correlates with probabilities and values —and the role of these variables can be tested empirically. These variables, in turn, are relevant to one of the brain processes that have been traditionally hardest to identify: selection of one among different possible behaviors.

Using decision theory and game theory to add to the conceptual tools at the disposal of the neuroscientist is Glimcher’s most singular contribution. Unfortunately, in the course of presenting it, Glimcher drags in so many red herrings that his contribution is easy to miss. Take a look at some of these alluring distractions.

MATHEMATICAL MODELING

Glimcher claims that his theory of how the brain selects a behavior is computational (a term he leaves us to interpret for ourselves), which introduces the concept of mathematical models in neuroscience. Until now, we have been discussing experimental neuroscience: trying to match a function with a brain mechanism by monitoring parts of the brain as we change the animal’s tasks or stimuli. Identifying phenomena experimentally is, of course, only one part of science.

To understand and quantify how a brain mechanism works, and make precise predictions, it is useful to capture what is observed in the lab in the form of models that can be studied mathematically. Simply put, a mathematical model is a mathematical formulation (such as an equation) that represents some aspect of the physical world.

This becomes indispensable when we move from understanding neurons to understanding complex systems in which millions of neurons act together. No one can keep track of the activity of all these neurons, let alone the interaction that gives rise to the brain’s most delicate functions, without appropriate mathematical tools. Forging these tools is the job of the growing field of theoretical neuroscience. 

Followed literally, this methodology risks building powerful models irrelevant to brain science because they ignore what is actually observed in the neurophysiologist’s lab. 

One neuroscientist who championed mathematical modeling was David Marr, best known for his mathematical theory of vision. Marr’s theory is eclectic, combining facts about the neurophysiology of vision, such as the role of certain cells in the retina, with almost any computational technique shown to be useful for processing images (including techniques unrelated to known neural processes). This did not bother Marr; he argued that to understand neural systems, theoretical neuroscientists should start by describing, as a computational problem, the function the system fulfilled. Once the computational problem had been formulated, scientists should guess at what series of intermediate steps (or what algorithm, in Marr’s terminology) would solve the problem. Then, and only then, might neuroscientists try to identify what neural mechanisms implemented the intermediate steps. Followed literally, this methodology risks building powerful models irrelevant to brain science because they ignore what is actually observed in the neurophysiologist’s lab. This was the fate of Marr’s theory of vision.

Still, many theorists followed Marr’s lead, and some successfully modeled neural processes. Glimcher insists that his own approach is computational, along the lines proposed by Marr, and he does exploit mathematical tools drawn from economics to model behavior. But he does not—at least in this book—follow Marr in creating mathematical models of the neural processes he is investigating.

A PUZZLING POLEMIC

Another of Glimcher’s themes is a polemic against the concept of reflexes. Suppose you observe that when enough heat is applied to the left hand, the left biceps contracts. This is an immediate response, which looks automatic, as though the organism had no choice but to withdraw the hand from the heat. This simple, direct response to a stimulus was called a reflex by the French philosopher and scientist René Descartes (1596-1650). He proposed that reflex behaviors were mechanically generated when the relevant stimuli acted on the organism’s body. Descartes distinguished reflexes from other behaviors, such as speaking, which he viewed as more complex. The organism might or might not elicit this more complex behavior—choosing to speak, for example, and what to say. This choosing was attributed by Descartes to a faculty called the will and from the Latin for will, voluntas, these are still often called voluntary behaviors. 

Neurophysiologists at the beginning of the 20th century who began recording neural activity, wondered—using our example of withdrawing the hand from heat— whether neurons whose activity varied with the application of heat were also responsible for exciting neurons that contract biceps. If so, this reflex behavior could be explained in mechanical terms (just as Descartes had predicted), through the action of the nervous system. These scientists identified circuits of neurons running from sensory receptors in the skin to the spine, and then back to muscular fibers, and claimed that these circuits generated many reflex behaviors. Conclusion: These reflexes can be generated by the peripheral nervous system and spine alone, without involving the brain. Thus, animals can do many things, even walk, without using their brains.

Glimcher argues that after Descartes, neuroscientists sought to reduce all behavior to reflexes and all neural mechanisms to mechanisms for reflexes. Glimcher then counters that the nervous system contains no reflex mechanisms. Instead, all behavior is explainable by his approach based on decision theory. This is the rhetoric of a paradigm shift in thinking about behavior, but rhetoric that is unwarranted and misleading.

To maintain, as Glimcher does, that most neuroscientists have attempted to reduce behavior to reflexes is, at best, an exaggeration. Most have believed that there is more to behavior than reflexes (namely, voluntary behavior), even if they had no tools for experimental study of the mechanisms underlying it. A glorious chapter of the history of neuroscience, after all, is the localization of mental functions in the brain. Consider the celebrated brain area discovered in the 19th century by Paul Broca, who ascribed to it the function of producing language. How many scientists ever maintained that linguistic behavior could be reduced to reflexes?

Usually, neuroscience textbooks restrict the term “reflex” to relatively simple neural circuits that generate specific movements in direct response to specific stimuli. The line between reflexive and nonreflexive neural mechanisms may be fuzzy, but there is consensus that high-level neural mechanisms have more complex response properties, interconnections, and control functions than do ordinary reflex mechanisms. Furthermore, what about the role of brain systems having to do with memory, problem solving, and emotions? These systems play no role in reflexes. Nowadays, no self-respecting neuroscientist would try to reduce voluntary behavior, such as language, to the same kind of circuitry that is responsible for the withdrawal reflex.

In the end, his polemic against reflexes is puzzling. His theory of how neural systems select among possible behaviors does not need to displace good old-fashioned reflexes in order to be a genuine contribution to one of the hardest areas of neuroscience.

That is not to say that reflexes do not exist. As behaviors, they can be observed, and the neural circuits underlying many reflex behaviors remain among the best understood in neuroscience. If Glimcher wants to convince his readers that there are no reflexes, as he repeatedly claims, he needs to do more. In the end, his polemic against reflexes is puzzling. His theory of how neural systems select among possible behaviors does not need to displace good old-fashioned reflexes in order to be a genuine contribution to one of the hardest areas of neuroscience. But it is not, as Glimcher may hope, a radical departure from contemporary neuroscience, one that he sees as offering the only mechanistic salvation from Cartesian dualism.

ATTACKING DUALISM

Descartes thought there was no brain or other mechanism for choosing voluntary behaviors. Such choices are left to the mind, a nonmechanical, nonphysical substance whose essence is conscious thinking. The mind can tell the body what to do once the mind has chosen, said Descartes, and the body can tell the mind what happens in and around it. But, this interaction notwithstanding, Descartes concluded that mind and body exist totally independently of each other.

This doctrine of two separate substances, called Cartesian dualism, faces insurmountable difficulties. First, no one has found convincing evidence that a nonphysical substance exists; and second, how would such a substance ever interact with the physical body (as Descartes believed it did)? In fact, Cartesian dualism has been rejected by many philosophers, who have been vindicated by the subsequent history of neuroscience. Decades of experiments with brains and brain lesions since the 19th century have shown how different brain areas are responsible for various aspects of mind. Some areas are necessary for the cognitive processing of sensory stimuli; without them, for example, we would be blind and deaf. Other areas are needed for complex behaviors; without them, we would lose control of our limbs. The remaining areas, as we suggested earlier, are assigned every other mental function: volition, emotion, problem solving, and so on. This assignment of mental functions to specific physical areas of the brain has dealt a big blow to Cartesian dualism. Today, neuroscientists and philosophers alike maintain that mind, including the control of voluntary behavior, is the product of the physical brain.

In his astonishing attack on Cartesian dualism, Glimcher seems to be conflating it with Descartes’s distinction between reflexive and voluntary behavior. He writes that Cartesian dualism is “the idea that two independent mechanisms would be required to account for human behavior.” Fair enough, as far as it goes, but Glimcher also refers to Cartesian dualism as the view that voluntary behavior is “the product of the soul,” and he purports to offer “an approach that may even allow us to resolve the dualism of body and soul.” These are clear references to Cartesian dualism.

By this logic, Glimcher is led to claim that until now, neuroscientists have had but two options for explaining voluntary behavior: They could follow Descartes in appealing to a nonphysical mind (or soul), or try to use the only mechanisms allowed by Descartes—reflexes. We have already seen, however, that Glimcher’s polemic against reflexes is misplaced, because, since the time of Descartes, neuroscience has identified mechanisms much more complex than reflexes, and these mechanisms are a way out of Glimcher’s purported dilemma. We do not need to choose between immaterial minds and reflex mechanisms; we already believe there is more to the brain than reflex mechanisms. Like his rejection of reflexes, Glimcher’s attack on Cartesian dualism confuses the reader and distracts from what is important in the book.

CHOOSING FREELY, CHOOSING AT RANDOM

Glimcher claims that his new kind of neural mechanism, understood through the lenses of decision theory and game theory, constitutes the first mechanistic theory of free will. This is his biggest blunder.

One of Descartes’s motivations for invoking a nonphysical mind was to make room for free will. The only apparent alternative—explaining voluntary behavior by some kind of neural mechanisms such as reflexes—seemed to Descartes to negate free will. “Free” is notoriously difficult to define. To a first approximation, however, free will is the power to choose what one wants. Choosing freely is often thought to be incompatible with determinism, which asserts that everything one does is necessitated by what came before. But free will is not the same as nondeterministic will. If one is bound to choose at random, with no power to influence what is chosen, then one’s will does indeed escape determinism; but where is its freedom? In fact, free will is not the same as absence of determinism. There is even a philosophical tradition, called compatibilism, according to which one’s will can be determined and yet free.

Glimcher repeatedly claims that his approach to the brain enables us for the first time to study free will scientifically. “One has such a clear sense…of exercising free will,” he writes, and he expects “tools like classical economics and the theory of games to bring decisions produced by free will under the umbrella of neuroscience.” Free will has always been an evanescent philosophical concept reserved for humans. Now Glimcher proposes to use the tools of neuroscience to find free will in monkeys. Remarkable.

Unfortunately, when push comes to shove, Glimcher is talking not about free will but about nondeterministic neural processes. He shows that game theory often requires organisms to be unpredictable to their opponents and describes studies showing that animals do behave in ways that are unpredictable to their external observers. Since unpredictability can be the result of a nondeterministic process, Glimcher concludes that animals select their behavior in a nondeterministic way and asserts that his approach based on game theory is the first to give a rigorous treatment of such choices.

The first problem is that none of this has much to do with free will. The second is that unpredictable behavior can, but need not, be generated by nondeterministic processes: It can also be generated by deterministic processes. A simple example is a computer randomizer, which spews out “random” quantities through a perfectly deterministic algorithm. The numbers are called random only because that is what they appear to be if we do not know the algorithm that generated them. All that game theory shows is that some optimal strategies require behavior that opponents are in no position to predict. It does not require that this behavior be generated by a nondeterministic process. Glimcher’s jump from game theory to nondeterministic behavior is unjustified.

In fairness, Glimcher does mention the distinction between “nondeterministic” and “unpredictable,” pointing out that game theory “does not require that animals be fundamentally unpredictable” (his italics). But this sounds like an afterthought, and Glimcher immediately adds that “devices as complex as our brains could have evolved the ability to produce random processes at the neuronal level.” He also says that some behaviors must be “irreducibly indeterminate” or “truly random.”

In the end, Glimcher’s theory fails as a contribution to debates about free will or Cartesian dualism, but succeeds as exciting neuroscience. 

In the end, Glimcher’s theory fails as a contribution to debates about free will or Cartesian dualism, but succeeds as exciting neuroscience. His exploration of the lateral intraparietal area improves our understanding of this part of the brain and does so by using ideas drawn from decision theory. This suggests that it is worthwhile exploring whether the same approach may illuminate other neural systems. If so, Glimcher’s may be a landmark contribution to understanding the elusive problem of how brains select behaviors.  

EXCERPT

From Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics, by Paul W. Glimcher. © 2003 by Paul Glimcher. Reprinted with permission of MIT Press.

What should a rational monkey be doing when he performs the cued saccade task? He should be trying to get as much Berry-Berry Fruit Juice as he can, as quickly as possible.

If we begin by assuming that the monkeys do have a goal, and that their goal is to maximize the juice they receive, then we ought to be able to use an economic approach to figure out how they should go about achieving that fairly straightforward goal. An economic approach would suggest that first, the monkeys would need to know the prior probability that looking at the upper target and looking at the lower target would yield rewards. Second, our monkeys would need to know the amount of juice that they could hope to receive for looking at either the upper or the lower target; they would need to know the value to each movement. Finally, our monkeys would have to combine an estimate of the prior probability of reward with an estimate of the value of each movement to determine something like the expected utility of each possible response. Then our monkeys would select and produce the movement with the higher expected utility.

We also realized that for a rational monkey, the expected utility for each movement would change as each trial progressed. Early in each trial, before the fixation light changed color, expected utility would be based on the prior probability that each movement would be rewarded, times the value (or, more precisely, the utility) of each movement. But after the fixation light changed color, the monkey could perform something like a Bayesian probability estimation to determine the posterior probability that looking at the upper or lower target would be rewarded. After the fixation light changed color, which in the cued saccade task indicated with 100 percent certainty which movement would be reinforced, the monkey could combine a posterior probability estimate with an estimate of value to produce a more accurate expected-utility estimate for each movement. Of course, in the experiment we had done, none of these variables, which were the only variables any economist would have considered worth varying, was ever manipulated.

ENCODING PROBABILITY Accordingly, we modified our cued saccade task to test a simple hypothesis... Since any rational decision-making system must encode the likelihood of all possible outcomes, we designed an experiment to ask if neurons in area LIP carry information about the probability of obtaining a reward. In all existing physiological studies of LIP, the likelihood that any movement would yield a reward had always been held constant. But if area LIP participated in solving the computational problem of deciding where to look, and if that computational problem could be solved rationally only by a system that kept track of probability, then the activity of neurons in area LIP might well be influenced by the likelihood that a movement would yield a reward.



About Cerebrum

Bill Glovin, editor
Carolyn Asbury, Ph.D., consultant

Scientific Advisory Board
Joseph T. Coyle, M.D., Harvard Medical School
Kay Redfield Jamison, Ph.D., The Johns Hopkins University School of Medicine
Pierre J. Magistretti, M.D., Ph.D., University of Lausanne Medical School and Hospital
Robert Malenka, M.D., Ph.D., Stanford University School of Medicine
Bruce S. McEwen, Ph.D., The Rockefeller University
Donald Price, M.D., The Johns Hopkins University School of Medicine

Do you have a comment or question about something you've read in CerebrumContact Cerebrum Now.