Playpen's Architecture and Behavior

Playpen's architecture and behavior are constrained both by the nature of its task and what is known about how language and vision operate.

1. As discussed in Sections 3.2.1 and 3.3 above, processing is interactive and bi-directional. Visual input can yield linguistic output, and linguistic input, along with some visual context, can yield visual output in the form of expectations or imagery.
2. In the pre-linguistic phase, the system receives visual inputs with no explicit labels, and learning consists of somehow extracting regularity from these inputs. In the linguistic phase, some visual inputs are accompanied by linguistic inputs as well. In one sense this just means that the input patterns are more complex; in another it means that there are now new independent grounds for dividing space up into particular categories.
3. Language is explicit about certain categories, in particular objects and relations. Learning language requires that the network have the capacity to represent these categories explicitly.
4. A good deal is known about the human vision system. While we believe there are large gaps that need to be filled in, the model should be in general agreement with the vision facts.
5. A good deal is known about language development and concept development. Playpen is meant to model the behavior of babies, and the most important constraints are those concerned with development: What precedes what? What is hard or easy?

We have paid most attention to the first four constraints. The fifth is likely to play a greater role as we model the development of particular relations.

• The Overall Architecture
• Processing Units
• Object Units
• Relation Units
  • The Problem
  • The Solution
• Sequential Time
• Learning