Evaluation of hyperbox neural network learning for classification

Mark Eastwood, Chrisina Jayne

Research output: Contribution to journalArticle

Abstract

This paper evaluates the performance of a number of novel extensions of the hyperbox neural network algorithm, a method which uses different modes of learning for supervised classification problems. One hyperbox per class is defined that covers the full range of attribute values in the class. Each hyperbox has one or more neurons associated with it, which model the class distribution. During prediction, points falling into only one hyperbox can be classified immediately, with the neural outputs used only when points lie in overlapping regions of hyperboxes. Decomposing the learning problem into easier and harder regions allows extremely efficient classification. We introduce an unsupervised clustering stage in each hyperbox followed by supervised learning of a neuron per cluster. Both random and heuristic-driven initialisation of the cluster centres and initial weight vectors are considered. We also consider an adaptive activation function for use in the neural mode. The performance and computational efficiency of the hyperbox methods is evaluated on artificial datasets and publically available real datasets and compared with results obtained on the same datasets using Support Vector Machine, Decision tree, K-nearest neighbour, and Multilayer Perceptron (with Back Propagation) classifiers. We conclude that the method is competitively performing, computationally efficient and provide recommendations for best usage of the method based on results on artificial datasets, and evaluation of sensitivity to initialisation.
Original languageEnglish
Pages (from-to)249-257
Number of pages9
JournalNeurocomputing
Volume133
Early online date14 Jan 2014
DOIs
Publication statusPublished - 10 Jun 2014

Fingerprint

Neurons
Learning
Neural networks
Supervised learning
Multilayer neural networks
Decision trees
Computational efficiency
Backpropagation
Support vector machines
Classifiers
Chemical activation
Decision Trees
Neural Networks (Computer)
Cluster Analysis
Efficiency
Weights and Measures
Datasets

Cite this

@article{a5dbc636ec6a45d5af631962e515aa54,
title = "Evaluation of hyperbox neural network learning for classification",
abstract = "This paper evaluates the performance of a number of novel extensions of the hyperbox neural network algorithm, a method which uses different modes of learning for supervised classification problems. One hyperbox per class is defined that covers the full range of attribute values in the class. Each hyperbox has one or more neurons associated with it, which model the class distribution. During prediction, points falling into only one hyperbox can be classified immediately, with the neural outputs used only when points lie in overlapping regions of hyperboxes. Decomposing the learning problem into easier and harder regions allows extremely efficient classification. We introduce an unsupervised clustering stage in each hyperbox followed by supervised learning of a neuron per cluster. Both random and heuristic-driven initialisation of the cluster centres and initial weight vectors are considered. We also consider an adaptive activation function for use in the neural mode. The performance and computational efficiency of the hyperbox methods is evaluated on artificial datasets and publically available real datasets and compared with results obtained on the same datasets using Support Vector Machine, Decision tree, K-nearest neighbour, and Multilayer Perceptron (with Back Propagation) classifiers. We conclude that the method is competitively performing, computationally efficient and provide recommendations for best usage of the method based on results on artificial datasets, and evaluation of sensitivity to initialisation.",
author = "Mark Eastwood and Chrisina Jayne",
year = "2014",
month = "6",
day = "10",
doi = "10.1016/j.neucom.2013.11.011",
language = "English",
volume = "133",
pages = "249--257",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Elsevier",

}

Evaluation of hyperbox neural network learning for classification. / Eastwood, Mark; Jayne, Chrisina.

In: Neurocomputing, Vol. 133, 10.06.2014, p. 249-257.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evaluation of hyperbox neural network learning for classification

AU - Eastwood, Mark

AU - Jayne, Chrisina

PY - 2014/6/10

Y1 - 2014/6/10

N2 - This paper evaluates the performance of a number of novel extensions of the hyperbox neural network algorithm, a method which uses different modes of learning for supervised classification problems. One hyperbox per class is defined that covers the full range of attribute values in the class. Each hyperbox has one or more neurons associated with it, which model the class distribution. During prediction, points falling into only one hyperbox can be classified immediately, with the neural outputs used only when points lie in overlapping regions of hyperboxes. Decomposing the learning problem into easier and harder regions allows extremely efficient classification. We introduce an unsupervised clustering stage in each hyperbox followed by supervised learning of a neuron per cluster. Both random and heuristic-driven initialisation of the cluster centres and initial weight vectors are considered. We also consider an adaptive activation function for use in the neural mode. The performance and computational efficiency of the hyperbox methods is evaluated on artificial datasets and publically available real datasets and compared with results obtained on the same datasets using Support Vector Machine, Decision tree, K-nearest neighbour, and Multilayer Perceptron (with Back Propagation) classifiers. We conclude that the method is competitively performing, computationally efficient and provide recommendations for best usage of the method based on results on artificial datasets, and evaluation of sensitivity to initialisation.

AB - This paper evaluates the performance of a number of novel extensions of the hyperbox neural network algorithm, a method which uses different modes of learning for supervised classification problems. One hyperbox per class is defined that covers the full range of attribute values in the class. Each hyperbox has one or more neurons associated with it, which model the class distribution. During prediction, points falling into only one hyperbox can be classified immediately, with the neural outputs used only when points lie in overlapping regions of hyperboxes. Decomposing the learning problem into easier and harder regions allows extremely efficient classification. We introduce an unsupervised clustering stage in each hyperbox followed by supervised learning of a neuron per cluster. Both random and heuristic-driven initialisation of the cluster centres and initial weight vectors are considered. We also consider an adaptive activation function for use in the neural mode. The performance and computational efficiency of the hyperbox methods is evaluated on artificial datasets and publically available real datasets and compared with results obtained on the same datasets using Support Vector Machine, Decision tree, K-nearest neighbour, and Multilayer Perceptron (with Back Propagation) classifiers. We conclude that the method is competitively performing, computationally efficient and provide recommendations for best usage of the method based on results on artificial datasets, and evaluation of sensitivity to initialisation.

U2 - 10.1016/j.neucom.2013.11.011

DO - 10.1016/j.neucom.2013.11.011

M3 - Article

VL - 133

SP - 249

EP - 257

JO - Neurocomputing

JF - Neurocomputing

SN - 0925-2312

ER -