Have you ever felt a sense of unease when looking at Sophia the Robot? Or have you tried to re-watch early 2000s CGI movies in the likes of Polar Express or Final Fantasy: The Spirits Within, only to find the animated characters just a bit… off? Or maybe you found yourself browsing YouTube at 3 AM and accidentally stumbled upon the classic creepy video titled I Feel Fantastic? Welcome to the realm of the uncanny valley.
What is the uncanny valley?
Figure 1. Mori’s classic model of the Uncanny Valley. (from Mori, 1970, 2012)
Figure 2. Bunraku puppet. (c) Savannah Rivka, licensed under CC BY-SA 4.0. While familiar and likeable to Mori, Western viewers would probably find it a bit more eerie.
The uncanny valley is a proposed phenomenon that describes the feeling of strangeness or creepiness evoked by artificial agents such as robots, dolls, or, more recently, computer-generated characters in visual media such as movies or video games. It was first described by Japanese roboticist Masahiro Mori in 1970. Mori proposed a non-linear relationship between the human likeness of an agent and the amount of positive affect towards the agent (see figure 1).
According to Mori, we are initially more likely to feel an affinity towards a clunky humanoid robot than an industrial mechanical arm. This affinity increases with human likeness until the agent is very similar to its human counterpart, but not identical. This could be an eerily lifelike prosthetic dummy, or a faceless mannequin. At this point, the similar, but not quite human appearance invokes a sudden dip in the affinity towards the agent, creating a descent in the graph: the eponymous uncanny valley.
Mori’s paper came out at a time when lifelike robots were purely in the realm of science fiction. Therefore, his model was purely hypothetical. Mori was more concerned with giving advice to designers of prosthetics about usability. His hypothesis fell into relative obscurity soon after publication, but its popularity resurfaced in the early 2000s when the original essay was first translated into English. Currently, in the age of robots like Sophia or Geminoid HI-1, and increasingly realistic, but still imperfect computer-generated visuals (the young Princess Leia in Rogue One comes to mind), the problem proposed by Mori is more relevant than ever.
The uncanny valley phenomenon remains a popular topic in the study and design of humanoid robotics and immersive technology. This type of research focuses mainly on identifying design characteristics that evoke the uncanny experience (Rosenthal-von der Pütten and Krämer, 2014). In order to fully understand the phenomenon, however, other researchers have instead investigated the underlying cognitive and neural mechanisms.
In this blog post, I will review different perspectives on the uncanny valley from cognitive neuroscience research.
Uncanny Valley and the predictive brain
Early neural evidence of uncanny valley-like responses came from a 2012 fMRI study conducted by Saygin and colleagues. They hypothesized that the uncanny valley is a side-effect of the brain’s tendency to generate theories about the world around us. Participants observed either: a human (see image C in figure 3), an android whose appearance was modelled after the human (B) or a mechanical skeleton of the android with its outer appearance removed (the Robot condition; A). The researchers were interested whether the neural response would be different between the incongruent Android condition (human-like appearance, mechanical movements) and the congruent Human and Robot conditions.
Figure 3. The robot, android, and human conditions. (from Saygin et al., 2012)
Indeed, whole brain repetition suppression analysis showed a distinct pattern of activation in comparison to the Human and Robot conditions. While the lateral temporal cortex was sensitive in all three conditions, there was additional activation of the parietal and frontal cortices when observing the incongruent Android. Further analyses confirmed stronger selectivity of the intraparietal sulcus to repetition in the Android condition. This region is primarily involved with action perception and perceptuo-motor coordination (see Grefkes & Fink, 2005).
How could these results be interpreted? According to the predictive coding perspective, stronger responses in the incongruent condition could indicate that the brain is working extra hard to “resolve” the error in prediction stemming from the mismatch between apperance and movement. However, as the authors themselves noted, the study was designed to be a more exploratory investigation of robot perception, rather than a direct analysis of the Mori’s hypothesized model. Crucially, Saygin and colleagues did not investigate the affective responses to uncanny, incongruent Androids.
Likeability and familiarity
A different approach was taken by Rosenthal-von der Pütten and colleagues (2019). Instead of purely studying the perceptual responses to non-human agents, they examined the uncanny valley from a more social and affective perspective. Participants rated humans and artificial agents to rate stationary visual stimuli showing humans and artificial agents on scales of likeability, familiarity, and human likeness. Additionally, their neural responses were measured with fMRI. The artificial agents ranged from computer-generated humans with exaggerated physical features and humanoid robots to robots with little resemblance to humans.
RP2, an example of a less-than-humanoid robot (c) Willow Garage, licensed under CC BY 3.0
In accordance with Mori’s hypothesis, likeability tended to increase with human likeness, but there was a sharp drop in the rating for artificial humans and some androids. The same pattern was shown in the BOLD response of the ventromedial prefrontal cortex (vmPFC) during the likeability rating, suggesting a key involvement of this region in the affective component of the uncanny valley effect. ROI and functional connectivity analysis during the decision-making task showed involvement of the vmPFC, the dorsomedial prefrontal cortex (dmPFC), and the temporoparietal junction (TPJ). These areas are strongly involved in social cognition, particularly social judgment, mentalising, and decision-making.
By more closely following Mori’s model and investigating both affective (familiarity, likeability) and perceptual (human likeness) dimensions of the uncanny valley, the authors found an involvement of a closely-connected network of areas underlying social information processing. These results extend the findings by Saygin et al. (2012), showing that the uncanny valley is not a purely cognitive process, but is closely connected to social and affective areas of the brain.
Conclusion and further reading
The studies by Saygin et al. (2012) and Rosenthal-von der Pütten et al. (2019) used contrasting methodologies and different theoretical frameworks to investigate the cognitive neuroscience underlying interactions with artificial agents. Nonetheless, the results of both studies indicate increased neural responses in certain areas in response to the most ambiguous and incongruent agents, demonstrating that the uncanny valley is a complex phenomenon resulting from the integration of predictive, perceptual, and social cognition-related networks of the brain. Back in 1970, when Masahiro Mori first wrote about the uncanny valley, androids that are nearly indistinguishable from real humans were purely in the realm of science fiction. However, in an era of increased reliance on technology, it is becoming clear that designers and roboticists cannot ignore the implications of the uncanny valley.
Saygin and colleagues (2012) summed up this problem nicely: “As human-like artificial agents become more commonplace, perhaps our perceptual systems will be re-tuned to accommodate these new social partners. Or perhaps, we will decide it is not a good idea to make them so closely in our image after all.”
In this blog post, I summarized two articles about the uncanny valley that seemed the most interesting to me as someone interested in the perceptual mechanisms behind the phenomenon. However, Mori’s hypothesis has been studied extensively from the perspective of numerous disciplines, and has been extended to not only to study robots and computer-generated characters, but has also been applied to sound design in horror games, and even analysis of Ancient Grecian art and mythology.
If you are interested in reading about the uncanny valley from a more developmental perspective, be sure to check out the article Is That Child Friends… with a Robot? Taking a Look from the Uncanny Valley by Ana Radanovič from Issue 12 of the ABC Journal.
References
Grefkes, C., & Fink, G. R. (2005). The functional organization of the intraparietal sulcus in humans and monkeys. Journal of Anatomy, 207(1), 3–17. https://doi.org/10.1111/j.1469-7580.2005.00426.x
Mori, M. (2012). The Uncanny Valley: The Original Essay by Masahiro Mori. IEEE Robotics & Automation Magazine. Retrieved from https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley
Rosenthal-von der Pütten, A. M., & Krämer, N. C. (2014). How design characteristics of robots determine evaluation and uncanny valley related responses. Computers in Human Behavior, 36, 422-439. https://doi.org/10.1016/j.chb.2014.03.066
Rosenthal-von der Pütten, A. M., Krämer, N. C., Maderwald, S., Brand, M., & Grabenhorst, F. (2019). Neural mechanisms for accepting and rejecting artificial social partners in the uncanny valley. The Journal of Neuroscience, 39(33), 6555–6570. https://doi.org/10.1523/JNEUROSCI.2956-18.2019
Saygin, A. P., Chaminade, T., Ishiguro, H., Driver, J., & Frith, C. (2012). The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Social Cognitive and Affective Neuroscience, 7(4), 413–422. https://doi.org/10.1093/scan/nsr025
.jpg){kind=link}
