|
Connectionism today generally refers to an approach in the fields of cognitive psychology, cognitive science and philosophy of mind which models mental or behavioral phenomena with neural networks, and is associated with a certain set of arguments for why this is a good idea.
Basic Principles
All modern connectionist models adhere to two major principles regarding the mind.
- Any given mental state can be described as a (n)-dimensional vector of numeric activation values over neural units in a network.
- Memory is created by modifying the strength or the architecture of the connections between neural units (hence the name connectionism). The connection strengths, or "weights", are generally represented as a (N×N)-dimensional matrix.
Connectionists generally stress the importance of learning in their models. As a result, many sophisticated learning procedures for neural networks have been developed by connectionists. Learning always involves modifying the connection weights. These generally involve mathematical formula to determine the change in weights when given sets of data consisting of activation vectors for some subset of the neural units.
By formalizing learning in such a way connectionists have many tools at their hands. A very common tactic in connectionist learning methods is to incorporate gradient descent over an error surface in a space defined by the weight matrix. All gradient descent learning in connectionist models involves changing each weight by the partial derivative of the error surface with respect to the weight. Backpropagation, first made popular in the 1980s, is probably the most commonly known connectionist gradient descent algorithm today.
Background
The prevailing form of connectionist models today is known as Parallel Distributed Processing (PDP). PDP form became popular in the 1980s with the release of Parallel Distributed Processing: Explorations in the Microstructure of Cognition - Volume 1 (foundations) & Volume 2 (Psychological and Biological Models), by James L. McClelland, David E. Rumelhart, and the PDP Research Group. PDP's roots are the perceptron theories from the 1950s. In spite of the fact that as early as 1952 Friedrich Hayek posited the idea of spontaneous order in the brain arising out of decentralized networks of simple units (neurons), Hayek's work was never cited in the literature of connectionism.
Another form of connectionist models is the Relational Network framework developed by the linguist Sydney Lamb in the 1960s. Relational Networks have only ever been used by linguists.
An earlier and rather different connectionistic view was held by Edward Thorndike, a turn of the century psychologist who studied learning, with his most famous contributions being work on how cats escaped from puzzle boxes, and his formulation of the Law of Effect. His analysis (and its descendants) are peppered with references to associations between stimuli and responses. Though the S-R aspect has today been abandoned by radical behaviorists and cognitive psychologists (including connectionists), it is easy to impose the notion of association and modification of association strength on connectionist models.
Connectionists are generally in agreement that recurrent networks (networks wherein connections of the network can form a directed cycle) are more like the human brain than feedforward networks (networks with no directed cycles). A lot of recurrent connectionist models incorporate dynamical systems theory as well. Many researchers, such as the connectionist Paul Smolensky (one of the authors of the original PDP books), have argued that the direction connectionist models will take is towards fully continuous, high-dimensional, non-linear dynamic systems approaches.
The Connectionist vs. Symbolist debate
Connectionists consider their approach to be in opposition to Symbolist approaches. The differences between symbolist and connectionist approaches are:
- Symbolists posit symbolic models that do not resemble underlying brain structure at all, whereas connectionists engage in "low level" modelling, trying to ensure that their models resemble neurological structures.
- Symbolists generally focus on the structure of explicit symbols (mental models) and syntactical rules for its internal manipulation, whereas connectionists focus on learning from environmental stimuli and storing this information in a form of connections between neurons.
- Symbolists believe that internal mental activity consists of manipulation of explicit symbols (see Language of thought), whereas connectionists believe that manipulation of explicit symbols are a poor model of mental activity.
See also
Literature
- Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, Kim Plunkett: Rethinking Innateness. A connectionist perspective on development. MIT Press, 1996
|
