Off-campus UMass Amherst users: To download dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users, please click the view more button below to purchase a copy of this dissertation from Proquest.
(Some titles may also be available free of charge in our Open Access Dissertation Collection, so please check there first.)
Grounding knowledge in sensors: Unsupervised learning for language and planning
Abstract
The physical world and the language that we use to describe it are full of structure. Very young children discover this structure with apparent ease. They somehow transform sensory information gathered while exploring their environment into knowledge that enables both successful planning and natural language communication, two of the defining characteristics of human intelligence. How might robots autonomously discover similarly useful structure in their sensor data? This dissertation describes a single computational model that accounts for the unsupervised discovery of both the fundamental units of natural languages—words and their meanings—and the fundamental units of plans—actions and their effects. At the core of the model is an algorithm called PERUSE that discovers recurring patterns in real-valued, multivariate time series. Given a set of time series containing acoustic data from spoken utterances, PERUSE discovers patterns that correspond to recurring words. Once the robot discovers words, a second algorithm makes it possible for the denotations of words to be learned from non-auditory sensor data about the robot's environment. The end result is a set of word/meaning pairs that allow the robot to make probabilistic judgments about the referents of words that it hears and about the chances of communicative success when using a word to describe its environment. When these algorithms are applied to sensor data collected while taking actions, the patterns discovered by PERUSE represent possible effects. The end result is a set of action/effect pairs that allow the robot to make probabilistic predictions about the results of taking actions from particular regions of continuous action spaces. The results of two sets of experiments are described. In the first, human subjects played with blocks in front of a robot and generated unrestricted natural language utterances to describe the blocks and their configurations. The system successfully discovered words and their denotations in three different languages—English, German and Mandarin Chinese. In the second experiment the robot sampled actions randomly from a continuous action space that determined its path of motion with respect to objects in its environment. The robot successfully discovered qualitatively distinct interactions as well as regions in its action space that reliably led to the various interactions.
Subject Area
Computer science
Recommended Citation
Oates, James Timothy, "Grounding knowledge in sensors: Unsupervised learning for language and planning" (2001). Doctoral Dissertations Available from Proquest. AAI3027237.
https://scholarworks.umass.edu/dissertations/AAI3027237