Future applications for personal robots motivate research into developing robots that are intelligent in their interactions with people. Toward this goal, in this paper we present an integrated socio-cognitive architecture to endow an anthropomorphic robot with the ability to infer mental states such as beliefs, intents, and desires from the observable behavior of its human partner. The design of our architecture is informed by recent findings from neuroscience and embodies cognition that reveals how living systems leverage their physical and cognitive embodiment through simulation-theoretic mechanisms to infer the mental states of others. We assess the robot's mindreading skills on a suite of benchmark tasks where the robot interacts with a human partner in a cooperative scenario and a learning scenario. In addition, we have conducted human subjects experiments using the same task scenarios to assess human performance on these tasks and to compare the robot's performance with that of people. In the process, our human subject studies also reveal some interesting insights into human behavior.
Arulampalam, M. et al. (2002). A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 50(2): 174-188.
2.
Atkeson, C.G. and Schaal, S. (1997). Robot learning from demonstration. Proceedings of the 14th International Conference on Machine Learning . Morgan Kaufmann, pp. 12-20.
3.
Baron-Cohen, S. (1991). Precursors to a theory of mind: Understanding attention in others. Natural Theories of Mind, Whiten, A. (ed.). Oxford, Blackwell , pp. 233-250.
4.
Barsalou, L.W., Niedenthal, P.M., Barbey, A. and Ruppert, J. (2003). Social embodiment. The Psychology of Learning and Motivation, Vol. 43. New York, Academic Press.
5.
Berlin, M., Gray, J., Thomaz, A. and Breazeal, C. (2006). Perspective taking: an organizing principle for learning in human-robot interaction. Twenty-First National Conference on Artificial Intelligence (AAAI 2006).
6.
Billard, A., Epars, Y., Calinon, S., Schall, S. and Cheng, G. (2004). Discovering optimal imitation strategies. Robotics and Autonomous Systems, 47: 69-77.
7.
Breazeal, C., Berlin, M., Brooks, A., Gray, J. and Thomaz, A.L. (2006). Using perspective taking to learn from ambiguous demonstrations. Journal of Robotics and Autonomous Systems , 54(5): 385-393.
8.
Breazeal, C., Buchsbaum, D., Gray, J. and Blumberg, B. (2005). Learning from and about others: toward using imitation to bootstrap the social competence of robots. Artificial Life , 11(1-2): 31-62.
9.
Brooks, A.G., Berlin, M., Gray, J. and Breazeal, C. (2005). Untethered robotic play for repetitive physical tasks. Proceedings ACM International Conference on Advances in Computer Entertainment.
10.
Carberry, S. (2001). Techniques for plan recognition. User Modeling and User-Adapted Interaction, 11(1-2): 31- 48.
11.
Carpenter, J., Clifford, P. and Fearnhead, P. (1999). Improved particle filter for nonlinear problems . Radar, Sonar and Navigation, IEE Proceedings, 146(1): 2-7.
12.
Cassimatis, N.L., Trafton, J.G., Bugajska, M.D. and Schultz, A.C. (2004). Integrating cognition, perception and action through mental simulation in robots. Journal of Robotics and Autonomous Systems, 49(1-2): 13-23.
13.
Cohen, P.R., Levesque, H.J., Nunes, J.H.T. and Oviatt, S.L. (1990). Task-oriented dialogue as a consequence of joint activity. Proceedings of the Pacific Rim International Conference on Artificial Intelligence, Nagoya, Japan.
14.
Davies, M. and Stone, T. (1995). Introduction. Folk Psychology: The Theory of Mind Debate, Davies, M. and Stone, T. (eds). Cambridge , Blackwell.
15.
Deak, G.O., Fasel, I. and Movellan, J. (2001). The emergence of shared attention: using robots to test developmental theories. Proceedings of the 1st International Workshop on Epigenetic Robotics.
16.
Demiris, J. and Hayes, G. (2002). Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model. Imitation in Animals and Artifacts, Dautenhahn, K. and Nehaniv, C. L. (eds). Cambridge, MA, MIT Press, 321-361.
17.
Downie, M. (2000). Behavior, animation, and music: The music and movemnt of synthetic characters. Master's Thesis, MIT, Cambridge, MA.
18.
Emond, B. and Ferres, L. (2001). Modeling the false-belief task: an ACT-R implementation of Wimmer & Perner (1983). Second Bisontine Conference for Conceptual and Linguistic Development in the Child Aged from 1 to 6 Years.
19.
Fasel, I., Deak, G.O., Triesch, J. and Movellan, J.R. (2002). Combining embodied models and empirical research for understanding the development of shared attention. Proceedings of the International Conference on Development and Learning.
20.
Gallese, V. and Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading . Trends in Cognitive Sciences, 2(12): 493-501.
21.
Gordon, R. (1986). Folk psychology as simulation. Mind and Language, 1: 158-171.
22.
Gorniak, P. (2005). The affordance-based concept. PhD Thesis, MIT, Cambridge, MA.
23.
Gray, J., Breazeal, C., Berlin, M., Brooks, A. and Lieberman, J. (2005). Action parsing and goal inference using self as simulator. 14th IEEE International Workshop on Robot and Human Interactive Communication (ROMAN). Nashville, TN, IEEE.
24.
Grosz, B., Hunsberger, L. and Kraus, S. (1999). Planning and acting together. The AI Magazine, 20(4): 23-34.
25.
Grosz, B.J. and Sidner, C.L. (1990). Plans for discourse. Intentions in Communication, Cohen, P. R., Morgan, J. and Pollack, M. E. (eds). Cambridge, MA, MIT Press, pp. 417- 444.
26.
Horvitz, E., Breese, J., Heckerman, D., Hovel, D. and Rommelse, K. (1998). The Lumiere Project: Bayesian user modeling for inferring the goals and needs of software users. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence , Madison, WI, 256-265.
27.
Intille, S.S. and Bobick, A.F. (1999). A framework for recognizing multi-agent action from visual evidence. AAAI-99. AAAI Press, pp. 518-525.
28.
Jenkins, O.C. and Matarić, M.J. (2002). Deriving action and behavior primitives from human motion data. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2551-2556.
29.
Johnson, M. and Demiris, Y. (2005). Perceptual perspective taking and action recognition . International Journal of Advanced Robotic Systems, 2(4): 301-308.
30.
Jones, H. and Hinds, P. (2002). Extreme work teams: using swat teams as a model for coordinating distributed robots. Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work. New York, ACM Press, 372- 381.
31.
Kalman, R. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1): 35- 45.
32.
Laird, J.E. (2001). It knows what you're going to do: adding anticipation to a quakebot. AGENTS '01: Proceedings of the 5th International Conference on Autonomous Agents. New York , ACM Press, pp. 385-392.
33.
Lashkari, Y., Metral, M. and Maes, P. (1994). Collaborative interface agents. Proceedings of the 12th National Conference on Artificial Intelligence, Vol. 1. Seattle, WA, AAAI Press .
34.
Lieberman, H. (ed.) (2001). Your Wish is My Command: Programming by Example. San Francisco, CA, Morgan Kaufmann.
35.
Meltzoff, A.N. (2005). Imitation and other minds: the "like me" hypothesis . Perspectives on Imitaion: From Neuroscience to Social Science, Vol. 2, Hurley, S. and Chater, N. (eds). Cambridge, MA, MIT Press, pp. 55-77.
36.
Meltzoff, A.N. and Decety, J. (2003). What imitation tells us about social cognition: a rapprochement between developmental psychology and cognitive neuroscience . Philosophical Transactions of the Royal Society: Biological Sciences, 358: 491-500.
37.
Movellan, J. and Watson, J. (2002). The development of gaze following as a Bayesian systems identification problem. Proceedings of the International Conference on Development and Learning.
38.
Nagai, Y., Asada, M., and Hosoda, K. (2002). Developmental approach accelerates learning of joint attention. Proceedings of the International Conference on Development and Learning.
Nicolescu, M.N. and Matarić, M.J. (2003). Natural methods for robot task learning: instructive demonstrations, generalization and practice . Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, Melbourne, Australia.
41.
Pollack, M. (1990). Plans as complex mental attitudes. Intentions in Communication, Cohen, P. R., Morgan, J. and Pollack, M. (eds). Cambridge, MA, MIT Press, pp. 77-103.
42.
Premack, D. and Woodruff, G. (1978). Does the chimpanzee have a theory of mind. Behavioral and Brain Sciences, 1: 515-526.
43.
Rao, A.S. and Murray, G. (1994). Multi-agent mental-state recognition and its application to air-combat modeling. Proceedings of the 13th International Workshop on Distributed Artificial Intelligence (DAI-94), Seattle, WA, pp. 283- 304.
44.
Rizzolatti, G., Fadiga, L., Gallese, V. and Fogassi, L. (1996). Premotor cortex and the recognition of motor actions . Cognitive Brain Research, 3: 131-141.
45.
Ronnie, L., Johansson, M. and Suzic, R. (2005). Particle filter-based information acquisition for robust plan recognition. Eighth International Conference on Information Fusion.
46.
Scassellati, B. (2001). Distinguishing animate from inanimate visual stimuli . Proceedings of the International Joint Conference on Artificial Intelligence.
47.
Scassellati, B. (2002). Theory of mind for a humanoid robot. Autonomous Robots, 12: 13-24.
48.
Schaal, S. (1997). Learning from demonstration. Advances in Neural Information Processing Systems, Vol. 9, Mozer, M., Jordan, M. and Petsche, T. (eds). Cambridge, MA, MIT Press, pp. 1040-1046.
49.
Tapus, A., Mataric, M. and Scasselatti, B. (2007). The grand challenges in socially assistive robotics . IEEE Robotics and Automation Magazine, 4: 35-42.
50.
Trafton, J.G., Cassimatis, N.L., Bugajska, M.D., Brock, D.P., Mintz, F.E. and Schultz, A.C. (2005). Enabling effective human-robot interaction using perspective-taking in robots. IEEE Transactions on Systems, Man, and Cybernetics, 35(4): 460-470.
51.
Trafton, J.G., Schultz, A.C., Perzanowski, D., Bugajska, M.D., Adams, W., Cassimatis, N.L., and Brock, D.P. (2006). Children and robots learning to play hide and seek. HRI '06: Proceeding of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction. New York, ACM Press, pp. 102-109.
52.
Wimmer, H. and Perner, J. (1983). Beliefs about beliefs: representation and constraining function on wrong beliefs in young children's understanding of deception. Cognition, 13: 103-128.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
