The Learning Rule

The specific learning rule used operates as follows. During training, a target is associated with each input pattern; this target represents the appropriate response to the input. In ordinary back-propagation, each output unit receives a target on each trial. But, as noted above, this is an implausible procedure, as it means that all possible responses which are not appropriate are punished. Further, as noted above, not all wrong answers are wrong in the same way and unlikely to be responded to the same way by parents. Accordingly, we give the network feedback for only two sorts of words, the correct word and any incorrect words to which the network has made a significant response. We defined a ``response threshold'' for the word units, 0.05 in all of the experiments reported on here; only activations above this threshold are treated as overt responses for which feedback is possible. Further, the target for these explicit errors depends on the input as follows.

1. The target for a correct response is +1.
2. For a response which is not a correct label for the input object under any circumstances (e.g., ``small'' for a large, red object), the target for the corresponding output unit is -1.
3. For a response which would be a correct label for the input object if it matched the lexical dimension input (e.g., ``large'' for a large, red object when the input question is ``what color is it?''), the target for the corresponding output unit is 0.