The Problem

We have seen how OUs permit the network to distinguish clusters of features from one another and how positive connections between noun OUs and non-linguistic ``semantic'' OUs can implement the binding that is part of understanding and producing nouns.

Now consider what would be required for relations, both relation terms and non-linguistic relations. A binary, non-reflexive relation relates two distinct objects, for example, a stick and a block in SUPPORT(BLOCK, STICK). To implement a relation in a distributed connectionist network, we first need to make two modifications to the standard view from predicate logic. First, rather than treat an expression like this as a predicate with a truth value, we will treat it as a set of correlations or inferences. In this sense, SUPPORT in the example means loosely that stick features and block features in particular relative positions correlate with one another. Viewed in terms of inference, the relation takes the form of a process of pattern completion, for example,

1. Given a supported stick, infer that the supporter is a block.
2. Given a block and stick in a support relationship, infer that the block is the supporter.

Note that the inferences are not absolute; whether they actually hold depends on a number of other factors, for example, whether the stick in question has a flat surface. In the network implementation, whether a particular relation holds would depend on the combined input coming into the relevant units and therefore on the entire network of other relational inferences in the system. Note also that from this connectionist perspective, there is no distinction between a relation between individual objects and a generalized relation over types of objects.

A second modification to the standard view of relations is to treat them as non-atomic. A distributed implementation of a relation involves multiple micro-relations, each relating a pair of features, one belonging to each of the two objects. A micro-relation represents a highly specific micro-inference. Thus knowledge about SUPPORT is made up of micro-relations specific to particular relative locations in some abstract representational space.

Within the Playpen framework described so far, we could represent a micro-relation as a negative connection between two OUs. If both units are activated, this connection causes them to repel each other. With different phase angles, they would then correspond to parts of different objects. Thus the micro-inference represented by such a connection would be: given whatever features are associated with unit A and whatever features are associated with unit B, the A features and the B features belong to two different objects. Recall, however, that a negative connection represents inhibition as well as repulsion. Thus the desired inference only holds to the extent that the two units are both activated, for example, when both have their activations clamped. When only the phase angles can change, the two units will tend to end up out-of-phase, indicating distinct objects. However, when both activation and phase angle are permitted to change, the negative connection can also result in one unit's inhibiting the other, the precise outcome depending on the initial state of the two units, the coupling function associated with the connection, and of course other weights and inputs into the units.

However, this way of representing relations gets us nowhere when it comes to correlations between relations. For example, we might want to represent the following inference: if A supports B, then A is probably larger than B. This involves an association between a SUPPORT relation and a BIGGER-THAN relation, thus in the network minimally four OUs, two for each relation. A further example is the association required to specify the meaning of a relation term: if ``A is on B'', then some object A' is on another object B'. This involves an association between a linguistic on relation and a non-linguistic ON relation. Any set of connections we set up among the four units necessary to two relations fails to capture the relationship between the relations that we want. Consider the connections shown in Figure 10. While the individual connections do seem to represent the phase relationships we want for this example, if we consider the connections separately, we see that they miss crucial conditions. Thus the positive connection between the HIGH unit in the LOCATION pair and the SMALL unit in the RELATIVE SIZE pair indicates that objects that are high tend to be small. But what we would like to convey is the more complex fact that objects that are higher than other objects tend to be smaller than the other objects. The relationship we want to represent requires that the micro-relations be treated as units, and the simple network shown in Figure 10 does not permit this.

 

 

Figure 10: Attempt at Representing Relation-Relation Correlations. Two units in each of two different layers on the Where side of the network are shown. Green connections have positive weights, red connections negative weights. This network offers no way to relate the location and size relations to one another directly.

Note that a conjunctive unit connected to both of the OUs in a particular micro-relation does not solve the problem either. While we could arrange for such a unit to turn on to the extent that the two input units are on (using positive connections which affect activation but not phase angle), the unit would fail to represent the relation because it has no way of holding onto the two phase angles of the inputs: the identity of the two objects in the relation is lost.

Apparently relations between relations require explicit units representing micro-relations and taking OUs and other relation units as inputs to their two role ``arms.'' The relation units (RUs) should

1. Become activated to the extent that both input (groups of) units are activated and the two (groups of) units are out-of-phase with each other.
2. Preserve the phase angles of the inputs, passing them on to other RUs.
3. Associate with other RUs via a mapping of the arms which preserves the appropriate phase angle relationships and also reflects the strength of the correlation between the two micro-relations.