Training neural networks using multiobjective particle swarm optimization - Yusiong, John Paul Tan (2006)
Neural Networks (NNs) exhibit interesting properties such as the ability to adapt, learn and generalize thus are said to be capable of "learning". Learning can be accomplished by training neural networks, which is a significant component in the field of supervised learning. To train a neural network, a training algorithm is employed. Multiobjective Particle Swarm Optimization-Crowding Distance (MOPSO-CD) is a recently formulated multiobjective algorithm that employs the mechanism of crowding distance computation. This study suggests an approach to neural network training through the simultaneous optimization of architectures and weights with a Particle Swarm Optimization (PSO)-based multi-objective algorithm. Most evolutionary computation-based training algorithms formulate the problem in a single objective manner by taking a weighted sum of the objectives from which a single neural network model is generated. Thus, the goal of this study is to determine whether MOPSO-CD can train neural networks involving two objectives: accuracy-Mean-Squared Error (MSE) and complexity-Minimum Description Length (MDL). Also, this study proposes rules for automatic removal of unnecessary neurons from the network based on the following idea: a connection is pruned if its weight is less than the value of the smallest bias of the network. Experiments performed on benchmark datasets taken from the UCI machine learning repository show that this approach provides an effective means for training neural networks that is competitive with other evolutionary computation-based training algorithms.