cob0013 (Photo credit: jonathanmccabe)
The development of language, memory and cognition and Connectionist models :
By Dr. Fawzy A. Osman Salama, BA., Bsc., Msc, Ph.D.,
Senior Consultant Clinical Neuropsychologist
CONTENTS ……………………………………….PAGE
Introduction…………………………………………………01
Conclusion ….. .. ………………………………..03
References ……………………………………………….03
Introduction
How useful are connectionist models for explaining the development of language, memory and cognition?
Within child development one can distinguish a variety of approaches that provide different methods and explanations of understanding the development of language, memory and cognition. There are three main approaches: the information processing, the neural network or connectionist and the functional approach. Models such as the modal model, the schema framework and semantic networks are all functional accounts, i.e., they attempt to describe what is happing in memory regardless what goes on in the brain.
Recently, connectionist models have been advanced. This opposes the information-processing accounts. It is more concerned with how the system develops. Connectionism is a concept based on theories of artificial intelligence and psycholinguistics, it is a model of how a system organizes linguistic input in such a way as to mimic the development seen in children’s language. The connectionist approach uses computer programmes designed to mimic the information processing in the brain. These systems often involve many interconnections between an input message and some form of output. The main differences between information processing and connectionist explanations concern the level and the process of information storage. For Information processing models, information stored as ‘symbols’ and as a pattern for connectionist models. Connectionism as model for explaining the development of language has as its starting point a human neural systems and it will be constructed in infants on the basis of what happens to the child’ senses. As more information becomes available the system will be extended and modified to incorporate the new material.
The connectionist model of Rumelhart and McClelland (1986) produced the same pattern of verb usage that is found in the language of young children. It was given information about verbs as its input and as required to produce past-tense verbs as its output. The model was able to produce this human-like behaviour without the necessity of learning a rule. Thus, although children’s language looks as if it is ‘rule-governed, perhaps there is no need for the complexity of rule knowledge at all. Although Rumelhart and McClelland findings have been hotly contested, the model appeared to make some of the same mistakes that children make in learning to speak (see for example, the longitudinal study Rescorla (1980) of children’s over-extension of words).
One claim of connectionism is that artificial neural networks are capable of learninghow to identify new stimuli; of forming new associations between units; and of extracting prototypes from a series of examples. Strauss, (1979) carried a research study on prototypes as averages examples of faces. He found that infant’s memory systems had extracted the prototypical face in less time than required. This result suggests that the more recognizable a face is to an infant, the less time it should be looked at for. Therefore, the memory which the infants had developed for faces seemed to be based on averaging the features of all the examples they had seen.
Most of the work on human memory has taken place within the information-processing paradigm. Memory can be divided into short-and long-term memory. Changes in the
Apparent size of short-term memory as children grow older seems to be due to changes in strategy rather than a general capacity increase.
Developments in the use of computational modelling techniques (McClelland and Rumelhart, 1985) allow us to be more confident that the functional model of working memory can be simulated computationally and might have neurological correlates. The model of conceptual memory (McClelland and Rumelhart, 1985) for example makes an important point about the relationship between prototypes (semantic memory) and episodic memory (individual examples), both are aspects of the same underlying process. Thus, children may have very complex stores of episodic information without having much semantic knowledge. Therefore, if researchers wish to look at very early memory, they should pay attention on episodic information, which is more likely to be preserved in infants. This adds significant theoretical rigour. But it is crucial to combine the computational modelling with experimental data from human participants. All too often, computational modelling progresses without direct reference to empirically derived data on human cognition.
Does connectionism offer a truly new scientific model or does it merely cloak the old notion of associationism as a central doctrine of learning and mental functioning?
Most recent associationist theories of development have proposed changes in connections among highly interconnected processing units. Such changes, it is claimed, can lead to the emergence (development) of highly sophisticated mental structures.
The interview between Bancroft and Plunkett explores differences between connectionist proposals for cognitive architecture and sorts of models that have traditionally been assumed in cognitive psychology. Plunkett claim that the major distinction is that, while both connectionist and classical architectures postulate representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinational syntactic and semantic structure. Accordingly, different parts of the brain do different types of task. Therefore, the knowledge is encoded in patterns of connectivity between neurones and that any change in our knowledge in some way reflected in changes in that pattern of connectivity. As a consequences, different structures or ‘architectures’, and by learning algorithms which give them unique properties for carrying out different types of task. Several arguments for combinatorial structures in mental representations are then discussed. These include arguments based on the ‘systematicity’ of mental representation: i.e., or the fact that cognitive capacities
always exhibit certain symmetries, so that the ability to entertain given thoughts with semantically related contents. Plunkett claim that such arguments make a powerful case that mind/brain is not connectionist at the cognitive level. He then consider the future position of connectionism and argue that what we have to internalise is indeed internalised by a process of learning and not by a process of inheritance. Plunkett and Bancroft then consider the possibility that connectionism may provide an account of neural (or ‘abstract neurological’) structures in which classical cognitive architecture is implemented. They survey a number of arguments that have been offered on favour of connectionism, and conclude that they are coherent only on this interpretation.
One major advantage of connectionism and all computer models of human cognition are that the model can be tested to see if it works in a similar way to humans. Some researchers believe they really are models of how the brain works. However, it is unclear enough to see a direction in which information flows in a connectionist network; i.e., how actually memory operates in the brain. By looking at the structure and organization of the memory system rather than its contents, we are biased towards producing explanations which view all people as similar. In fact, differences between people are differences in their memories.
To sum up, we are still a long way away from understanding exactly what memory is, and how it works.
Conclusion:
Connectionist models of language, memory and cognition development are an important areas of research activity, with some promising results along side with the other functional and earlier models. It is more likely that a new theoretical position incorporating all the theories and models available will be produced.
References:
1-Rescorla, L. (1980). Overextension in early language development. Journal of Child Language, 7, pp. 321-35.
2-Rumelhart, D. E. and McClelland, J. L. (1980). On learning the past tenses of Englishverbs in McClelland, J. L. and Rumelhart, D. E. (eds) Parallel Distributed Processing: explorations in the microstructure of cognition. Vol. 2. Psychological and Biological Models. Cambridge, Mass., MIT Press.
The development of language, memory and cognition and Connectionist models :
By Dr. Fawzy A. Osman Salama, BA., Bsc., Msc, Ph.D.,
Senior Consultant Clinical Neuropsychologist
CONTENTS ……………………………………….PAGE
Introduction…………………………………………………01
Conclusion ….. .. ………………………………..03
References ……………………………………………….03
Introduction
How useful are connectionist models for explaining the development of language, memory and cognition?
Within child development one can distinguish a variety of approaches that provide different methods and explanations of understanding the development of language, memory and cognition. There are three main approaches: the information processing, the neural network or connectionist and the functional approach. Models such as the modal model, the schema framework and semantic networks are all functional accounts, i.e., they attempt to describe what is happing in memory regardless what goes on in the brain.
Recently, connectionist models have been advanced. This opposes the information-processing accounts. It is more concerned with how the system develops. Connectionism is a concept based on theories of artificial intelligence and psycholinguistics, it is a model of how a system organizes linguistic input in such a way as to mimic the development seen in children’s language. The connectionist approach uses computer programmes designed to mimic the information processing in the brain. These systems often involve many interconnections between an input message and some form of output. The main differences between information processing and connectionist explanations concern the level and the process of information storage. For Information processing models, information stored as ‘symbols’ and as a pattern for connectionist models. Connectionism as model for explaining the development of language has as its starting point a human neural systems and it will be constructed in infants on the basis of what happens to the child’ senses. As more information becomes available the system will be extended and modified to incorporate the new material.
The connectionist model of Rumelhart and McClelland (1986) produced the same pattern of verb usage that is found in the language of young children. It was given information about verbs as its input and as required to produce past-tense verbs as its output. The model was able to produce this human-like behaviour without the necessity of learning a rule. Thus, although children’s language looks as if it is ‘rule-governed, perhaps there is no need for the complexity of rule knowledge at all. Although Rumelhart and McClelland findings have been hotly contested, the model appeared to make some of the same mistakes that children make in learning to speak (see for example, the longitudinal study Rescorla (1980) of children’s over-extension of words).
One claim of connectionism is that artificial neural networks are capable of learning
how to identify new stimuli; of forming new associations between units; and of extracting prototypes from a series of examples. Strauss, (1979) carried a research study on prototypes as averages examples of faces. He found that infant’s memory systems had extracted the prototypical face in less time than required. This result suggests that the more recognizable a face is to an infant, the less time it should be looked at for. Therefore, the memory which the infants had developed for faces seemed to be based on averaging the features of all the examples they had seen.
Most of the work on human memory has taken place within the information-processing paradigm. Memory can be divided into short-and long-term memory. Changes in the
Apparent size of short-term memory as children grow older seems to be due to changes in strategy rather than a general capacity increase.
Developments in the use of computational modelling techniques (McClelland and Rumelhart, 1985) allow us to be more confident that the functional model of working memory can be simulated computationally and might have neurological correlates. The model of conceptual memory (McClelland and Rumelhart, 1985) for example makes an important point about the relationship between prototypes (semantic memory) and episodic memory (individual examples), both are aspects of the same underlying process. Thus, children may have very complex stores of episodic information without having much semantic knowledge. Therefore, if researchers wish to look at very early memory, they should pay attention on episodic information, which is more likely to be preserved in infants. This adds significant theoretical rigour. But it is crucial to combine the computational modelling with experimental data from human participants. All too often, computational modelling progresses without direct reference to empirically derived data on human cognition.
Does connectionism offer a truly new scientific model or does it merely cloak the old notion of associationism as a central doctrine of learning and mental functioning?
Most recent associationist theories of development have proposed changes in connections among highly interconnected processing units. Such changes, it is claimed, can lead to the emergence (development) of highly sophisticated mental structures.
The interview between Bancroft and Plunkett explores differences between connectionist proposals for cognitive architecture and sorts of models that have traditionally been assumed in cognitive psychology. Plunkett claim that the major distinction is that, while both connectionist and classical architectures postulate
representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinational syntactic and semantic structure. Accordingly, different parts of the brain do different types of task. Therefore, the knowledge is encoded in patterns of connectivity between neurones and that any change in our knowledge in some way reflected in changes in that pattern of connectivity. As a consequences, different structures or ‘architectures’, and by learning algorithms which give them unique properties for carrying out different types of task. Several arguments for combinatorial structures in mental representations are then discussed. These include arguments based on the ‘systematicity’ of mental representation: i.e., or the fact that cognitive capacities
always exhibit certain symmetries, so that the ability to entertain given thoughts with semantically related contents. Plunkett claim that such arguments make a powerful case that mind/brain is not connectionist at the cognitive level. He then consider the future position of connectionism and argue that what we have to internalise is indeed internalised by a process of learning and not by a process of inheritance. Plunkett and Bancroft then consider the possibility that connectionism may provide an account of neural (or ‘abstract neurological’) structures in which classical cognitive architecture is implemented. They survey a number of arguments that have been offered on favour of connectionism, and conclude that they are coherent only on this interpretation.
One major advantage of connectionism and all computer models of human cognition are that the model can be tested to see if it works in a similar way to humans. Some researchers believe they really are models of how the brain works. However, it is unclear enough to see a direction in which information flows in a connectionist network; i.e., how actually memory operates in the brain. By looking at the structure and organization of the memory system rather than its contents, we are biased towards producing explanations which view all people as similar. In fact, differences between people are differences in their memories.
To sum up, we are still a long way away from understanding exactly what memory is, and how it works.
Conclusion:
Connectionist models of language, memory and cognition development are an important areas of research activity, with some promising results along side with the other functional and earlier models. It is more likely that a new theoretical position incorporating all the theories and models available will be produced.
References:
1-Rescorla, L. (1980). Overextension in early language development. Journal of Child Language, 7, pp. 321-35.
2-Rumelhart, D. E. and McClelland, J. L. (1980). On learning the past tenses of Englishverbs in McClelland, J. L. and Rumelhart, D. E. (eds) Parallel Distributed Processing: explorations in the microstructure of cognition. Vol. 2. Psychological and Biological Models. Cambridge, Mass., MIT Press.
Fawzy's lucky people 