IAC network as an example: one should get a feeling for the behavior of the system behavior: balance of `forces'; there are many indirect as well as direct excitation paths; dynamics often exhibits slow buildup; stability is achieved by attractor basins.

NETWORKS. In order to make them useful, one needs:

1. lots of parameters - `complex systems'
2. balancing mechanisms - (eg, normalization of inputs) to retain sensitivity
3. organization that is functional - that works, adapts, that KNOWS, that models stuff
4. adaptation at ALL time scales is essential for competitive life:

a. centuries/millenia -- by genetics
b. individual lifetime -- learning, physical adaptation
c. year/hour/second - action, behavior, vegetative functions (eg, heart, hair growth)

Networks of various kinds can do various parts of these. LEARNING is adaptation at the timescale of an individual lifetime. (Perhaps we could say that ACTION is adaptation at the time scale of seconds, hours and years.)

The invariance of the organism is that of physical structure - our human shape and internal construction. The Body-Gene-Body cycle means that we can contribute a little bit of ourselves (if we are lucky) to the next generation. The (near) invariance of connection structure over time in the brain implies that cognitive function must respect approximate physical continuity. Thus learning rules must be local in general (though they work better if they are not completely local).

ARCHITECTURE of network models.

Input units -usually relatively more sensory (closer to the periphery) in canonical models.
Output units - usually closer to a perception or an action.
Hidden units - internal units (neither input nor output)
Activation rule - an equation summing excitation and inhibition
Output Rule - output to other units (derived from activation)

Some examples:

1. Pattern Associator. A set of input nodes and a set of output nodes. The system moves toward a simple pattern on the output nodes (eg, a single node being at 1 while the others are close to zero). Thus, they might be, eg, 10 input nodes mapped onto 2 output nodes (eg, a binary decision), or perhaps a thousand -> hundred. Hebb's rule for learning.

2. Backprop Network. 2 layers permit non-linearly separable recognition
Eg, the Bottleneck Network (eg, with 20 input nodes, 5 Hidden nodes and 20 output nodes). Useful for discovering a small set of `features'.

3. Interactive Activation (IAC) - for associative memory (Jets and Sharks). Partial info permits full retrieval and `remindings' etc.

4. Specialized processors. Eg, Grossberg and Rudd's `apparent motion' system

5. Adaptive Resonance Model: F1 (input STM), F2 (category STM), LTM, resonance state, mutual competition, Reset.

R. Port, Q500, 1999