| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

Prose Proposal

Page history last edited by PBworks 16 years ago

 

Cognitive Robotics

 

Goals and Objectives:

 

We aim to create a robot that is capable of navigating through the world, create an internal representation of the world it is in, know that it is a part of this representation, and identify a limited number of action schemas in the world. An action schema in this context is defined as a preposition, such as “between” and “under.” We chose these two action schemas as a starting point.

 

In his article Natural Semantics for a Mobile Robot, Paul Cohen et. al. describes his approach to this problem and the obstacles he has faced when dealing with this semantic issue. Here, I will summarize his approach and propose what we plan to do differently to solve the problems he posits. Cohen focuses on a few key points in achieving the goal I have outlined above. He describes the difference between functional semantics and natural semantics, examines the problem of agency and assigning roles to the environment, and states the difficulty of programming syntax into a robot.  In this essay, I will outline each of these issues and state how I propose to solve them. In my summary of Cohen’s article and in describing our own project, I will draw upon the cognitive theories of Lawrence Barselou. The theories I will incorporate are Image Schema and Perceptual Symbols Systems.

 

Traditional robots have been programmed in such a way to enable them to have functional semantics. In this paradigm, if a robot avoids running into the wall, then it is said to “know” the wall is there. However, the robot has just been programmed to avoid objects that its sensors tell it are too close. The robot does not know that it is a thing in the world and needs to avoid running into other things in order to function optimally. The feedback it provides does not carry any meaning to the robot “all meanings are our meanings; the machine has no independent understanding of anything.”[1] The machine appears to be acting intelligently, but in reality, it is only following a pre-programmed routine and does not make any on-line judgments about the meaning of its actions. This issues leads into the domain of agency.

 

In order for the robot to have natural semantics, it would need to know that it is an agent interacting with objects in the world. A natural semantic system “is one that acquires and maintains meanings for itself.”[2] This is how humans interact with the world; we build up a knowledge base of categories and action schemas and learn new concepts in the world based upon these previously constructed domains. Humans do not have a specific set of pre-programmed responses. We make decisions based upon how they will impact us and our desired outcome of a situation.  A robot would need to do a similar thing in order for it to be considered a natural semantic system; it would have to know that it is the beneficiary of its actions. One way to achieve this goal is to enable the robot to assign roles to objects in the environment. The robot would gather information about its environment and assign roles relative to how the robot is able to physically manipulate the object and how it can interact with the object overall. After many encounters, it would gradually update the role assignment and create a semantic network that is inherently meaningful to it because it would be based upon its own interaction with the object rather than a pre-programmed action routine.

 

This way of viewing cognition has roots in Lawrence Barsalou’s theory of Perceptual Symbol Systems and Image Schemata. In his article Percecptual Symbol Systems, Barsalou defines this approach to cognition, and its logic may give insight into programming natural semantics into a robot. Cohen proposes to employ this theory of cognition as a base in solving the problem of natural semantics in the future.

When one interacts with the world, the body is the primary means of this interaction. Barsalou proposes that there are primitive schematics of this interaction that he calls image schema. Starting in infancy, the mind abstracts a basic representation of how the body interacts with the world. [3]Only some of the information is used and as one repeats the action, the image schema continues to develop and eventually serves as the base of conceptual thinking. [4]For example, BALANCE (image schema are conventionally capitalized) would give rise to metaphorical thinking about being able to balance things in life, the balance between good and evil etc.[5] These abstract concepts are, he claims, rooted in the body’s interaction with the world[6]. The brain extracts certain details from the neural instantiation of the action and “schematizes” them.[7] This schema serves as a base for thought as is evident in metaphorical thinking. Cohen utilizes this line of thinking and notices that a robot would physically interact with the environment in similar ways as humans. For example, we both move on a horizontal plane, move things by pushing, things enter and exit our visual field, and we experience resistance when pushing against an object.[8] Therefore, Cohen proposes that if one programs the image schemas into the robot, then the robot will have a basis for thinking in terms of how its body interacts with the world. The meanings that it assigns to the world will have to be acquired and maintained by the robot in terms of its own body system, a natural semantic system.

 

Cohen has come up with a theoretical solution to the problem, but he fails to describe or achieve actual implementation of the proposal. The project I am envisioning addresses these issues of agency.

 

Project Description:

 

In our theoretical approach to this problem, we have concentrated on the idea of having two separate but complimentary perceptual systems. One of these systems is the actual data the robot is receiving, and the other is the representation space. The ability to identify action schemas within the second representation space is central to our notion of a natural semantic robot. The robot will be mobile and programmed with a certain curiosity that prompts him to explore the environment. It will create a map of that environment with the multiple sensors that are included with the robot kit we are planning to use, Lego Mindstorm NXT. It will constantly be updating that map; when it moves it will rescan the environment and compare its new scan with recent scans to distinguish if it is the same environment. When the robot undergoes a certain action schema, such as “between,” it will trigger a separate linguistic program running wirelessly from a nearby computer that will display a picture of one ball between two objects. The image will be a schematic representation of “between.”  This computer screen will be the representation space for the robot. This is the first step towards natural semantics because the computer will be able to display semantic information about its environment. However, with only this feature, the robot would still be classified as employing functional semantics because the information it is representing is meaningful to us, but the robot still doesn’t know that it is an agent interacting with the two entities.

 

For this goal, the robot needs agency; there has to be feedback from the representation space to the robot. The robot needs to have itself represented in that space and get information about itself and its surrounding environment.  To reach this goal, we have two things to accomplish. We need the robot to assign roles to the environment, and a feedback loop for the representation space. This feedback loop from the representation space is where our project diverges from that of Cohen. We will work to employ his theoretical research of robotics and image schema, and in addition we hope to contribute substantially to this idea by implementing this dual route of perception model.

The roles a robot assigns have to be related to how the robot can interact with them, as stated by Cohen in his image schema research. But in order for the robot to know that it is interacting with anything, to know that it is the thing interacting, to have a sense of agency, it must receive feedback from its representation space informing it that it is existing in this space with these objects and the body can manipulate them in this way. With this feedback feature, pre-programmed rudimentary image schema, and a program that will recognize action schema’s in the environment, the robot will have all it needs to start constructing a natural semantic system.

 

The crucial question is then how will this feedback loop work? The answer may lie in a relatively simple virtual robot program. This program called BLANK was designed BLANK and is able to manipulate virtual objects in its environment by typed command from the user. The robot manipulates the space and remembers labels attributed to new configurations. If this program is synched with the robot in reality, then we may have a solution to our problem. Instead of simply remembering the virtual steps taken to manipulate the object, the program could store the embodied actions the real robot used in order to manipulate the object. In this way, this program could serve as a complex internal representation of the space the robot is in and provide the robot with meaningful feedback that it can use to manipulate objects in novel ways and create a natural semantic system based upon this relationship. The synched program would begin with certain image schemas and action schemas in place. The machine could then cluster its experiences in a way that aligns with Barsalou’s theory of Perceptual Symbol Systems.

 

In this paradigm, Barsalou proposes that during the process of perceptual redescription, a process of extracting important details from the environment for storage to be used in later situations, somewhat akin to that of image schema production, but across all sensory modalities instead of just how the body interacts with the environment. In order to understand this system, the concepts of frames and domains has to be defined. A domain is a general body of knowledge that is activated to provide a base of information that the current situation is understood relative to. A frame is a context which elicits a particular domain. For example, a domain of things that roll is activated for the frame of car wheel. When this domain activates, it serves as an umbrella for all of the previous experiences across all modalities which pertain to this situation.[9] For example, when I experience a car I look at the visual aspects, the color and shape, it has a particular smell, it may be exhilarating to drive, I may have had certain positive or negative experiences of car crashes etc. All of these are considered perceptual symbols. Certain elements of each perception are schematized and added to the Perceptual Symbol System for car.[10] Each time one has an experience, the perceptual symbol system for that domain is partially updated. This network of information is built up over time in order to produce coherent concepts that derive from lived experience with the concept in question. The robot can structure its knowledge in this way to acquire natural semantics. All of the experiences it has with an object will be subsumed under categories that are created based upon interaction with the object. This system will then carry inherent meaning because they will be structured and organized according to the robots own experiences.   

 

Included in the robots perceptual symbol system will be the action schemas and image schemas derived from both the physical interaction with the environment and representation of these events in the virtual space. This will provide the robot with a holistic system that will serve as a base for acquiring natural semantics.

 

This is the overarching goal of the project at hand. However, in order to achieve this goal, incremental steps have to be taken. The first steps will include a pilot project starting this semester as a class project. The class, Cognition and Computation, is taught by Professor Per Aage Brandt, who will be supervising this project. The pilot will include building a robot from the relatively cheap and rudimentary Lego Minstorm NXT. Using the Lego kit will allow a pilot robot to be built by the end of the semester. However, sacrificing complexity for time will put limitations on the abilities of the robot. With this pilot we will strive to achieve an internal representation for the robot which will be driven by the sensors provided with the kit. We also aim to link this robot wirelessly to the linguistic program in order to display the image file of an action schema when the robot provides information that triggers a threshold value of the action schema. For example if it is within six inches of two objects on either side of it, the screen will read “between” with the corresponding image. Working alongside this project will be another group in the class designing the virtual robot to be used in the later stages of this project. This group’s goal will be to have this virtual robot done by the end of the semester. I plan on continuing this research into the summer along with a few other students from the class. In this stage we will work on acquiring a more complex robot that is suited for the overarching goal of this project.

 

 



[1] Paul Cohen, Carole Beal, “Natural Semantics for a Robot,” (1999), http://citeseer.ist.psu.edu/cohen99natural.html

[2] Ibid.

[3] Robert L Goldstone and Lawrence W Barsalou, “Reuniting Perception and Conception,” Cognition, no.65 (1998) 231-262

[4] Ibid.

[5] Ibid.     

[6] Ibid.

[7] Ibid.

[8] Paul Cohen, Carole Beal, “Natural Semantics for a Robot,” (1999), http://citeseer.ist.psu.edu/cohen99natural.html

[9] Lawrence W. Barsalou, “Perceptual Symbol Systems,” Behavioral and Brain Scienes, no. 22 (1999): 577-660

[10] Ibid.

Comments (0)

You don't have permission to comment on this page.