Behavioural science: secret signals
You answer the telephone and hear a perky voice:"Hello, this is Michelle from Brown's computers. I wonder if you have a few moments to talk about some of our new products?"
Sales call! Your first impulse is to slam the receiver down. But -- well, at least the woman is polite. And knowledgeable. And...
Fifteen minutes later, as you give the ever-cheerful Michelle your credit-card number to purchase a new Super Zippy hard drive, you tell yourself what a good deal it is, and how much you've been needing the thing.
Or do you? Computer scientist Alex Pentland from the media lab at the Massachusetts Institute of Technology (MIT) in Cambridge would argue that your decision had very little to do with what the saleswoman said, and everything to do with how she said it -- her intonation, say, or the fluctuating pace and amplitude of her voice. The sale was probably made within the first few seconds, he says, guided by signals you did not even perceive.
"Human behaviour is much more predictable than is generally thought," says Pentland. A person's responses can often be explained by "non-linguistic behaviours of other people and simple instincts for social display and response, without any recourse to conscious cognition".
This 'second channel' of human communication acts in parallel with that based on rational thinking and verbal communication, and it is much more important in human affairs than most people like to think, says Pentland. Yet evidence for it has been accumulating in the laboratory for decades. And now modern technology has made it possible to monitor these social signals in spontaneous, natural settings on a scale that was not possible before.
"Collecting this kind of spontaneous data is extremely hard," says neuroscientist Anna Esposito of the International Institute for Advanced Scientific Studies in Vietri sul Mare, Italy. "But Alex has found a way to do it."
"Human behaviour is much more predictable than is generally thought."
Alex Pentland
Indeed, over the past decade Pentland and his team have developed a range of wearable devices that include electronic badges, modified personal digital assistants (PDAs) and specially configured mobile phones. These instruments can track not only a person's location, but also his or her behavioural details -- such as intonation and upper-body movements. The resulting data can then be analysed for patterns of meaningful social signals.
Pentland and his colleagues have already done multiple experiments with this technology at call centres and elsewhere, and are now moving into sites such as banks and hospitals, where they collect data on hundreds of people over many months to reveal the hidden communications in organizations. The potential reward, Pentland says, is a richer, more complete and more objective view of human interaction -- with our inherent bias toward what's conscious and verbal taken out of the equation.
Out of control
"It is difficult for people to accept", says psychologist John Bargh of Yale University in New Haven, Connecticut, a pioneer in the study of subliminal influences on human behaviour, but much of a person's everyday life "is determined not by their conscious intentions and deliberate choices, but by mental processes put into motion by their environment".
"A gazillion experiments show that I can flash something at you so fast you don't see it, yet the information does bias you towards one decision as opposed to another," says cognitive psychologist Michael Gazzaniga of the University of California at Santa Barbara. Moreover, he says, the literature is full of experiments showing that conscious explanations for our behaviour are often just rationalizations invented after the fact. He cites the example of a patient whose corpus callosum had been severed as a treatment for epilepsy, making it impossible for one side of the brain to communicate with the other. Gazzaniga and his colleagues presented the word 'walk' to the patient's left visual field, which corresponds to the right side of the brain. When the patient stood up and began walking, they asked him why; the right side of the patient's brain had been shown to lack the ability to process language. His left brain, which never received the walk command, but which handles language processing, quickly invented a logical explanation: "I wanted to go get a Coke."
Analogous effects have been found many times in healthy people. In the early 1990s, for example, psychologists Robert Rosenthal of the University of California, Riverside, and Nalini Ambady of Tufts University in Medford, Massachusetts, asked college students to give reasons for liking or disliking their instructors. The students tended to mention the instructors' friendliness or the clarity of their lectures, attributing their rating to conscious assessment. But Rosenthal and Ambady found that most of this is simply rationalization. The researchers could predict how around 70% of the students would rate an instructor just by analysing the instructor's body language in 30 seconds of soundless video (N. Ambady and R. Rosenthal. J. Pers. Soc. Psychol. 64, 431-441; 1993).
Pentland's efforts to track such phenomena in the wild, so to speak, date back to the early 2000s. In one experiment, the group took data over a period of several months in 2005, as operators at a call centre in the United Kingdom tried to make sales over the telephone. The operators were fitted with small electronic devices that measured variations in their tone and pitch, but not the specific words they used. Yet the researchers were able to devise an algorithm that could predict whether a call would result in a sale from only a few seconds of data. Successful operators, it turned out, spoke little and listened more. And when they did speak, their voices fluctuated strongly in amplitude and pitch, suggesting interest and responsiveness.
"If operators do it right," says Pentland, "they are almost certain to be successful." Indeed, the call centre now uses this new understanding to train its operators to sell more effectively, and to recruit people who have the right speech patterns. Other experiments suggest that such measures could improve sales success by as much as 20%, says Pentland, which is worth millions to large industries.
And in an experiment involving a 45-minute mock salary negotiation between students in a business school, Pentland says that by combining several display signals from the first 5 minutes of the negotiation, his team could predict who would come out on top with 87% accuracy.
As a result of such experiments, the MIT group has identified a handful of common social signals that predict the outcomes of sales pitches, the success of bluffing in poker, even subjective judgements of trust. These signals include the 'activity level', effectively the fraction of time the person speaks; their 'engagement' or how much a person drives the conversation; and 'mirroring', which occurs when one participant subconsciously copies another's prosody and gesture.
Pentland is well aware that not everyone has embraced this vision of a hidden, pervasive 'second channel' in human affairs. Although the idea is reasonably well accepted for other animals, Pentland says that the reluctance to accept it in humans lies in part because of an inherent bias in how science looks at human behaviour versus that of other species. "If our data were collected from ape troops," he says, "and we altered the semantically loaded labels a bit, talked about 'forage' instead of 'work', 'food access' instead of 'salary', they would feel entirely unsurprising."
Keep it simple
Behavioural science, as he sees it, ought to seek the simplest explanation for human behaviour first, looking to simple social signals, before constructing more complicated explanations based on language and conscious reasoning. Indeed, he says, humans lived in social groups long before language evolved, and the language function presumably exists on top of a more archaic brain system for non-linguistic social signalling.
The behaviour of non-human primates supports this idea, says primatologist Frans de Waal of Emory University in Atlanta, Georgia. Apes, chimpanzees and other primates -- our close evolutionary cousins -- lack anything like our facility for language, yet still lead sophisticated social lives through displays of power, meaningful noises and facial expressions.
So it is "incredibly naive", he says, to take conscious verbal communication as the primary way that people respond to each other.
In the future, Pentland hopes, the type of work he and his colleagues have been doing can be expanded to look in much more detail at human social interactions in larger groups. In an ongoing experiment at a large German bank, for example, they are using their sensors to focus on personal characteristics that are thought to be linked to group creativity. According to recent work on creativity, says computer scientist Peter Gloor, who studies collaborative innovation networks at MIT's Sloan School of Management and who has collaborated with Pentland, creative people tend to be more open and agreeable, and less neurotic than others, and they tend to fluctuate more between extroversion and introversion. These traits can be assessed with standard psychological questionnaires, but Pentland's group thinks the sensors can do the job more effectively, giving managers more reliable ways to identify patterns of human behaviour, and thereby to assemble the best possible creative teams.
"Collecting this kind of spontaneous data is extremely hard. But Alex has found a way to do it."
Anna Esposito
The group had 22 bank employees wear data-collecting badges for one month, and studied how the data revealed personal characteristics relevant to creativity. For example, from the sensor data, the team was able to construct a 'contribution index' for each person that measured, roughly speaking, their pro-activeness in interpersonal interactions. Someone with contribution index of 1 looks frequently at other people. Someone with an index of -1 is only ever looked at, he or she never looks another actor squarely in the face. The group then compared this measure with descriptors of each individual as assessed in a standard psychological test, judging variables such as the person's 'openness' towards fantasy, aesthetics, feelings and new ideas, or their 'extroversion', spontaneous warmth, gregariousness, assertiveness and activity.
High values of the index, they found, correlated very strongly with the psychological assessment of extroversion. "The more one person looks at others," says Pentland, "the more of an extrovert they seem to be."
Pentland agrees that this is the key challenge, but suggests that the sensors are orders of magnitude more powerful than what was used to study human non-verbal interactions in the past. "People used to measure mostly short-term features such as nods or smiles, whereas we can make continuous, quantitative measurements over much longer times," he says. "We can also examine precise details of frequency and amplitude, something you can't do easily without automated tools."
The earlier work, he suggests, was akin to trying to understand written text by looking just at individual letters, ignoring words or sentences. Moreover, he suggests, almost all previous work was done in the lab, and quantification was all but impossible in anthropological-style field work.
"In contrast," Pentland says, "we are taking people in real situations, in day-to-day work settings over months, and finding lots of reliable and directly predictive relationships."
One thing he does admit is that experiments of this sort raise the spectre of Big-Brother-style intrusiveness. Pentland hopes that corporations won't look at the technology as just another way to spy on employees, which would almost guarantee resentment and loss of morale. He says that many of the problems could be addressed if the companies followed the protocol used in his own group's experiments: the data were first stored locally on each subject's machine, so that at the end of the day, he or she could review what had been collected and decide whether it would be shared or permanently deleted. Pentland also says that employees should be given the ability to turn off the device if they wish.
If these kinds of protections are respected, he says, then the devices could lead to fundamentally better science, including a deeper knowledge of that second channel of human social interaction. "People have been studying this kind of stuff for a long time," says Gazzaniga, "but Alex is raising it up to another level. It is very clever stuff."