Variable hidden layer sizing in feedforward and Elman recurrent neuro-evolution
Date
1998-08
Authors
Garlick, Ryan
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Artificial neural networks are learning systems composed of layers of neurons, modeled after the human brain. The relationship between the size of the hidden layer in a neural network and performance in a particular domain is currently an open research issue. Often, the number of neurons in the hidden layer is chosen empirically, and subsequently fixed for the training of the network. Fixing the size of the hidden layer limits an inherent strength of neural networks - the ability to generalize experiences from one situation to another, to adapt to new situations, and to overcome the "brittleness" often associated with traditional artificial intelligence techniques. This thesis proposes an evolutionary algorithm to search for network sizes that exhibit good performance, along with weights and connections between neurons. The size of the networks simply becomes another search parameter for the evolutionary algorithm. This allows for faster development time, and is a step toward a more autonomous learning system.
This thesis builds upon the neuro-evolution tool SANE, developed by Risto Miikkulainen and David Moriarty. SANE stands for symbiotic adaptive neuro-evolution and is a novel learning system proven extremely effective in a range of problems. SANE is modified in this work in several ways, including varying the hidden layer size and evolving Elman recurrent neural networks for enhanced performance. These modifications allow the evolution of better performing and more consistent networks, and evolve more efficiently and faster - in every domain tested.
This performance enhancement is demonstrated in two real-world applications. First, SANE, modified with variable network sizing, learns to play modified casino blackjack and develops a successful card counting strategy. Second, these modifications are applied to an agent in a simulated search and obstacle avoidance environment.
The contributions of this research are performance increases in a decision strategy generation system and a more autonomous approach to the scaling of neuro-evolutionary techniques for solving larger and more difficult problems.
Description
Keywords
neural networks, computer algorithms, artificial intelligence
Citation
Garlick, R. (1998). Variable hidden layer sizing in feedforward and Elman recurrent neuro-evolution (Unpublished thesis). Southwest Texas State University, San Marcos, Texas.