Variable Hidden Layer Sizing in Elman Recurrent Neuro-Evolution
Date
2000-05
Authors
Kaikhah, Khosrow
Garlick, Ryan
Journal Title
Journal ISSN
Volume Title
Publisher
Kluwer Academic Publishers
Abstract
The relationship between the size of the hidden layer in a neural network and performance in a particular domain is currently an open research issue. Often, the number of neurons in the hidden layer is chosen empirically and subsequently fixed for the training of the network. Fixing the size of the hidden layer limits an inherent strength of neural networks the ability to generalize experiences from one situation to another, to adapt to new situations, and to overcome the brittleness often associated with traditional artificial intelligence techniques. This paper proposes an evolutionary algorithm to search for network sizes along with weights and connections between neurons. The size of the networks simply becomes another search parameter for the evolutionary algorithm.
This research builds upon the neuro-evolution tool SANE, developed by David Moriarty. SANE evolves neurons and networks simultaneously, and is modified in this work in several ways, including varying the hidden layer size, and evolving Elman recurrent neural networks for non-Markovian tasks. These modifications allow the evolution of better performing and more consistent networks, and do so more efficiently and faster.
SANE, modified with variable network sizing, learns to play modified casino blackjack and develops a successful card counting strategy. The contributions of this research are up to 8.34% performance increases over fixed hidden layer size models while reducing hidden layer processing time by almost 10%, and a faster, more autonomous approach to the scaling of neuro-evolutionary techniques to solving larger and more difficult problems.
Description
Keywords
neuro-evaluatory, Elman recurrent neural networks, Computer Science
Citation
Kaikhah, K., & Garlick, R. (2000). Variable hidden layer sizing in Elman recurrent neuro-evolution. Applied Intelligence, 12(3), pp. 193-205.