About Faces, in Art and in the Brain

Faces and Survival

Almost all creatures, from dogs to canaries, gorillas to penguins, salamanders to crocodiles, kangaroos to spiders, and certainly members of our human family, manifest the same formula for facial composition: a forehead, two eyes, a nose, a mouth, and a chin that are in the same relative positions. Why was nature so consistent in composing faces? Essentially, the arrangement optimizes survival: the eyes located high for a commanding view of the world, the nose turned down to avoid the rain, and the mouth situated to ingest food that has been perused by the nose and eyes above it.

Faces are usually the first thing we notice in other people. Faces tell us more about a person than any other physical attribute and have always been socially important to humans. More often than not, we evaluate each other from first impressions based largely on facial expressions. In general, faces we find attractive or trustworthy, we bond with; faces we find offensive or deceitful, we avoid. Just below this behavioral act, however, lies a more profound reason for our response to faces.

Throughout the long course of human evolution, recognizing and evaluating people on the basis of their faces made an important contribution to survival. Knowing whom to trust and whom to suspect saved many of our ancestors from needless bloodshed, so they could live to obtain food and mate another day. As cooperative societies emerged and people relied on each other for working together in hunting, food gathering, or constructing shelter, it was helpful to rely on bodily and facial expressions as much as on words to comprehend intentions and emotions. The formation of alliances, vitally important in what we might call the “hive-life” of social people, depended on communicating trustworthiness, which could be done, in part, through the always conspicuous face. Ability to read faces became a tool for human survival.

Charles Darwin’s classic book, The Expression of the Emotions in Man and Animals,¹ pointed out that facial gestures, as well as some postures, were part of the survival kit of some animals. Discussing aggressive expressions of animals, Darwin wrote: “When a dog is on the point of springing on his antagonist, he utters a savage growl; the ears are pressed closely backwards, and the upper lip is retracted out of the way of his teeth, especially of his canines.” Species from menacing cobras to laughing baboons to play-acting birds such as the common killdeer (which tries to draw predators away from its nest with cries of “killdeee, killdeee” and a “broken wing” charade) have survived another day by demonstrating aggressive or distracting behaviors.

The Universality of Facial Expression

Human facial expressions are also designed to arouse reactions from others. The early work of Darwin (who took full advantage of the British Empire’s broad geography) indicated that people all over the world both express and perceive facial expression in similar ways. These early observations were confirmed recently by Paul Ekman, Ph.D., of the University of California Medical School at San Francisco, who enlarged Darwin’s observations, using more sophisticated methods.^2,3 Traveling to exotic locations such as Papua New Guinea where the residents had little contact with outsiders, Ekman showed the natives photographs of people expressing anger, happiness, disgust, surprise, sadness, and fear and asked them to say what emotion was being expressed. He found wide general agreement in the emotions identified, which he interpreted as meaning “our evolution gives us universal expressions, which tell others some important information about us.” In these universal signals we are able to read another’s emotions, attitudes, and truthfulness. Of course, exceptions abound, such as the everyday deceptions put forward for our benefit by, for example, actors on stage and salespeople who have learned to mimic genuine facial expressions. Although culture shapes the nuances of facial expressions, nature seems to endow us with a limited number of basic emotions and ways to convey them, much like a language, which is basically the same everywhere it is used but is subject to local dialects.

Learning from Face Blindness

If nature has furnished us with special adaptive skills for facial expression, how are these skills manifested in our brains? Do we have neural systems just for reading faces? Both clinical medicine and cognitive neuroscience can help to answer these questions.

Since ancient times, physicians noticed that some patients exhibit a type of visual agnosia (a-gnosis, “non-knowledge”) in which visual information in general is not understood. Related to this problem is another more specialized condition in which a person is unable to process facial information. The 19th-century French neurologist Jean Martin Charcot recognized a type of “face blindness” in an otherwise healthy person who had difficulty recognizing a close friend by his face. In the mid-20th century, this disorder was given the name “prosopagnosia,” which literally means “facial non-knowledge.”

In classic cases of prosopagnosia, the person’s behavior is striking. He notes and understands all other types of sensory information—such as what type of perfume someone might be wearing, what color scarf she has on, and the sound of her voice— yet, cannot recognize even a close relative by that person’s face. One celebrated case of prosopagnosia is reported by the physician Oliver Sacks, M.D., in his popular book The Man Who Mistook His Wife for a Hat.⁴ Sacks’s patient could recognize his wife if she wore a well-known hat or some other familiar item, but, by facial information alone, he was mystified as to the identity of this nice lady visitor.

Prosopagnosia’s narrow and specific deficit suggested to scientists that either a specialized part of the brain was involved in facial perception or some general impairment existed in the way the brain put together perceptual components to compose the gestalt of an object such as a face. A first hypothesis was that prosopagnosia was caused by brain lesions, trauma, or damage to the temporal lobe. We now know that the neurologic reason for this selective blindness is that a cluster of specialized cortical cells vital for facial processing are damaged, making normal facial identification difficult but sparing vision for other things (such as a hat). Scientists learned through postmortem examinations and, more recently, neuroimaging studies that a specialized part of the right hemisphere is involved in facial processing. We can understand this better by looking at how visual information in general is processed by the brain.

Visual Processing in the Brain

Understanding the neuronal pathways involved in vision—from the eye to the brain to the achievement of object recognition—once seemed terribly complicated, engaging a multitude of perceptual and cognitive elements. Now, we have a good overview of this visual processing system. It begins when sensory information from the eye enters the primary visual cortex in the posterior part of the brain, activating a series of neurons upward in the dorsal stream and downward in the ventral stream. Each pathway has a distinct function in processing. The dorsal stream largely identifies where an object is; the ventral stream is involved in identifying what an object is. In the ventral stream, forms, colors, and faces are found. We call this process “the streaming of visual information,” although the metaphor of streams is an oversimplification of a brain teeming with cross-circuit communication in which object identification and the rules of processing are intricately entwined.

On the hilly topography of the brain is a region called the occipitotemporal cortex, which processes visual information. Located within that region is a specific outcropping called the fusiform gyrus, or the fusiform facial area (FFA). By means of measuring regional cerebral blood with positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) scans, scientists have discovered that this part of the brain is active when people look at faces. The exact nature of the FFA is still being studied, with some researchers hypothesizing that faces are conceptualized as holistic configurations, while others suggest that faces are processed as a composite of features.

This divergence of opinion is part of a larger debate in neuroscience. On one hand, it has been suggested that the brain consists of domain-specific modules that are specialized for carrying out explicit functions, such as processing sensory events (in this case, faces)—the so-called “Swiss Army knife” model. Others, however, argue that the brain consists of more general mechanisms that are broad enough to handle a variety of different processing operations (for example, faces but also other visual signals). Faces, according to this latter view, would acquire their special status through repeated exposure. The considerable evidence gathered using neuroimaging, specialized electroencephalograms, and other techniques to study the processing of faces and face facsimiles favors the domain-specific hypothesis: faces are unique and seem to occupy a distinct cerebral space or module.

Inverted Faces

Look at the face to the right. Do you recognize this popular figure? Now, turn the page and see how much easier it is to recognize him when the picture is upright.

This phenomenon is called the facial inversion effect. Because inverted faces convey the same information as upright faces— all features and relationships are identical, only the orientation differs—one might expect similar visual processing patterns and cerebral involvement. But experiments show that different cortical locations are involved in processing inverted faces, providing further evidence that the FFA is a module dedicated to the processing of upright faces. Frank Tong, Ph.D., showed subjects a series of upright and inverted faces and objects similar to the ones below and used fMRI to measure cerebral blood flow, an indication of neurologic activity.⁵ Looking at the upright faces produced a considerable increase in cerebral blood flow, whereas the inverted cartoon of Mickey Mouse and the cow produced less activity.

Curiously, people with prosopagnosia tend to do relatively better identifying inverted faces than upright ones, which suggests that upright normally viewed faces are processed by a different cluster of neurons than are inverted faces. Processing inverted faces seems similar to processing other complex visual stimuli, for example, a painting of a pastoral scene by Monet— that is, it appears to involve a partial decomposition of a whole face into components.

Further evidence that upright faces are processed by a specialized brain module, while inverted faces are processed by an analysis of components, is found in the “Thatcher Effect,” so called because a particularly off-putting picture of the former prime minister of Britain was used in an experiment that demonstrated it. Here, we illustrate the effect below with Tom Cruise. At first glance, there is nothing peculiar about this picture, but, if you turn it right side up, you will see it is grotesque.

Inside the Artist’s Mind

The depiction of human faces has been a theme of artists from ancient Egyptian wall paintings, to early Hebrew and Christian representations, to Renaissance portraits, to modern abstract paintings. Some contemporary artists present us with distorted faces such as the disturbing portrait by Francis Bacon on the next page. This “face” stymies our psychological need to understand the emotion being conveyed as well as processing by the neurologic cluster in our brain’s FFA to make sense of what we see. In frustrating our normal propensity to understand and process, Bacon forces us to look deeper into the picture.

We know little about the way an artist’s brain works while painting a face—although, if attention to faces involves increased blood flow to the FFA, then common sense would tell us that artists throughout the ages must have experienced a veritable flood in that brain region. Although facial perception is associated with increased blood flow to the FFA, expert facial artists may actually show less blood flow as contrasted with novice painters.

Consider a famous artist such as Raphael, arguably the most important portrait painter of the Renaissance. What cerebral activity might be going on as he composed Madonna of the Meadow (1506)? From recent discoveries in cognitive neuroscience, it is possible to recreate the activity of Raphael’s brain as well as the activity of those who view this classic piece of art.

Look first at the exquisite faces painted by Raphael. Now, consider the areas of the brain activated by various components of the painting. A visual scene is viewed bit-bybit as our eyes focus on one part and then move (in little jumps called “saccades”) to another point, and so on. In the case of Madonna of the Meadow, one might focus on the face of Mary, then on the colors, then on a geometric form, and then on cues to perspective, such as distance. Each of these features has specific loci in the cortex as shown in the dorsal (“where”) and the ventral (“what”) streams. Note the location of Mary’s face in the FFA.

Although looking into Raphael’s brain with the use of modern imaging techniques is impossible, we can apply such techniques to today’s artists. Several years ago, I undertook a project called “The Artist’s Brain” to see what cerebral structures might be involved in the production of art. Initially, I tested the feasibility of the study by modifying an MRI machine so I could copy a portrait of a face while having my brain scanned. The results indicated that successful data could be collected this way, and a detailed study was launched in which a leading portrait painter drew a series of faces while undergoing fMRI analysis.

Humphrey Ocean, named by the National Portrait Gallery in London as one of the foremost portrait artists of the 20th century, consented to having his brain scanned as he drew a series of faces. Measures of his brain activity6 were contrasted with a control subject who had no particular training in art. The results are shown on the next page. It appears that a novice artist requires greater cerebral “effort,” as indicated by increased regional cerebral brain flow in the FFA than does an experienced portrait painter, who spends hours each day over years looking at and analyzing faces. Perhaps Ocean is so well practiced at facial perception that he is less likely than a novice to ponder the features and gestalt of a face. Furthermore, if Ocean’s brain is especially efficient at processing faces, he may be able to allocate more cerebral effort to deeper aspects of a person’s face. My preliminary results did indicate that Ocean showed greater activation in the right frontal area (see upper right two scans) than did the novice painter, which suggests that the expert painter used “higher order” cognitive processing. In effect, he could be “thinking” a face, as well as “seeing” it.

 

The Important Function of Faces

Our faces, which we take so much for granted as we shave or apply makeup each morning, are a powerful and privileged component of our nature, with an important role in evolution, social life, and art. For centuries, artists implicitly understood this significance of the face by arousing our perceptions and reactions with portraits of all kinds. With modern techniques we are now beginning to understand how the brain’s structure and operation reflect the centrality of faces in daily lives—a lesson first learned many thousands of years ago on the Serengeti Plains as early humans grappled with urgent issues of self-preservation based on the non-linguistic gestures of other humans.