Overlap-Based Undersampling for Improving Imbalanced Data Classification

Pattaramon Vuttipittayamongkol; Eyad Elyan; Andrei Petrovski; Chrisina Jayne

doi:10.1007/978-3-030-03493-1_72

Overlap-Based Undersampling for Improving Imbalanced Data Classification

Pattaramon Vuttipittayamongkol, Eyad Elyan, Andrei Petrovski, Chrisina Jayne

Centre for Digital Innovation

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Citations (Scopus)

Abstract

Classification of imbalanced data remains an important field in machine learning. Several methods have been proposed to address the class imbalance problem including data resampling, adaptive learning and cost adjusting algorithms. Data resampling methods are widely used due to their simplicity and flexibility. Most existing resampling techniques aim at rebalancing class distribution. However, class imbalance is not the only factor that impacts the performance of the learning algorithm. Class overlap has proved to have a higher impact on the classification of imbalanced datasets than the dominance of the negative class. In this paper, we propose a new undersampling method that eliminates negative instances from the overlapping region and hence improves the visibility of the minority instances. Testing and evaluating the proposed method using 36 public imbalanced datasets showed statistically significant improvements in classification performance.

Original language	English
Title of host publication	Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings
Editors	Hujun Yin, Paulo Novais, David Camacho, Antonio J. Tallón-Ballesteros
Publisher	Springer-Verlag
Pages	689-697
Number of pages	9
ISBN (Print)	9783030034924
DOIs	https://doi.org/10.1007/978-3-030-03493-1_72
Publication status	Published - 2018
Event	19th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2018 - Madrid, Spain Duration: 21 Nov 2018 → 23 Nov 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11314 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2018
Country/Territory	Spain
City	Madrid
Period	21/11/18 → 23/11/18

Bibliographical note

Publisher Copyright:
© 2018, Springer Nature Switzerland AG.

Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.

Access to Document

10.1007/978-3-030-03493-1_72

Cite this

Vuttipittayamongkol, P., Elyan, E., Petrovski, A., & Jayne, C. (2018). Overlap-Based Undersampling for Improving Imbalanced Data Classification. In H. Yin, P. Novais, D. Camacho, & A. J. Tallón-Ballesteros (Eds.), Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings (pp. 689-697). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11314 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-03493-1_72

Vuttipittayamongkol, Pattaramon ; Elyan, Eyad ; Petrovski, Andrei et al. / Overlap-Based Undersampling for Improving Imbalanced Data Classification. Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings. editor / Hujun Yin ; Paulo Novais ; David Camacho ; Antonio J. Tallón-Ballesteros. Springer-Verlag, 2018. pp. 689-697 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{14021afb0d08464696de7922e675fcf7,

title = "Overlap-Based Undersampling for Improving Imbalanced Data Classification",

abstract = "Classification of imbalanced data remains an important field in machine learning. Several methods have been proposed to address the class imbalance problem including data resampling, adaptive learning and cost adjusting algorithms. Data resampling methods are widely used due to their simplicity and flexibility. Most existing resampling techniques aim at rebalancing class distribution. However, class imbalance is not the only factor that impacts the performance of the learning algorithm. Class overlap has proved to have a higher impact on the classification of imbalanced datasets than the dominance of the negative class. In this paper, we propose a new undersampling method that eliminates negative instances from the overlapping region and hence improves the visibility of the minority instances. Testing and evaluating the proposed method using 36 public imbalanced datasets showed statistically significant improvements in classification performance.",

author = "Pattaramon Vuttipittayamongkol and Eyad Elyan and Andrei Petrovski and Chrisina Jayne",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG. Copyright: Copyright 2019 Elsevier B.V., All rights reserved.; 19th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2018 ; Conference date: 21-11-2018 Through 23-11-2018",

year = "2018",

doi = "10.1007/978-3-030-03493-1_72",

language = "English",

isbn = "9783030034924",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "689--697",

editor = "Hujun Yin and Paulo Novais and David Camacho and Tall{\'o}n-Ballesteros, {Antonio J.}",

booktitle = "Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings",

}

Vuttipittayamongkol, P, Elyan, E, Petrovski, A & Jayne, C 2018, Overlap-Based Undersampling for Improving Imbalanced Data Classification. in H Yin, P Novais, D Camacho & AJ Tallón-Ballesteros (eds), Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11314 LNCS, Springer-Verlag, pp. 689-697, 19th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2018, Madrid, Spain, 21/11/18. https://doi.org/10.1007/978-3-030-03493-1_72

Overlap-Based Undersampling for Improving Imbalanced Data Classification. / Vuttipittayamongkol, Pattaramon; Elyan, Eyad; Petrovski, Andrei et al.
Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings. ed. / Hujun Yin; Paulo Novais; David Camacho; Antonio J. Tallón-Ballesteros. Springer-Verlag, 2018. p. 689-697 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11314 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Overlap-Based Undersampling for Improving Imbalanced Data Classification

AU - Vuttipittayamongkol, Pattaramon

AU - Elyan, Eyad

AU - Petrovski, Andrei

AU - Jayne, Chrisina

PY - 2018

Y1 - 2018

N2 - Classification of imbalanced data remains an important field in machine learning. Several methods have been proposed to address the class imbalance problem including data resampling, adaptive learning and cost adjusting algorithms. Data resampling methods are widely used due to their simplicity and flexibility. Most existing resampling techniques aim at rebalancing class distribution. However, class imbalance is not the only factor that impacts the performance of the learning algorithm. Class overlap has proved to have a higher impact on the classification of imbalanced datasets than the dominance of the negative class. In this paper, we propose a new undersampling method that eliminates negative instances from the overlapping region and hence improves the visibility of the minority instances. Testing and evaluating the proposed method using 36 public imbalanced datasets showed statistically significant improvements in classification performance.

AB - Classification of imbalanced data remains an important field in machine learning. Several methods have been proposed to address the class imbalance problem including data resampling, adaptive learning and cost adjusting algorithms. Data resampling methods are widely used due to their simplicity and flexibility. Most existing resampling techniques aim at rebalancing class distribution. However, class imbalance is not the only factor that impacts the performance of the learning algorithm. Class overlap has proved to have a higher impact on the classification of imbalanced datasets than the dominance of the negative class. In this paper, we propose a new undersampling method that eliminates negative instances from the overlapping region and hence improves the visibility of the minority instances. Testing and evaluating the proposed method using 36 public imbalanced datasets showed statistically significant improvements in classification performance.

UR - http://www.scopus.com/inward/record.url?scp=85057084930&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-03493-1_72

DO - 10.1007/978-3-030-03493-1_72

M3 - Conference contribution

AN - SCOPUS:85057084930

SN - 9783030034924

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 689

EP - 697

BT - Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings

A2 - Yin, Hujun

A2 - Novais, Paulo

A2 - Camacho, David

A2 - Tallón-Ballesteros, Antonio J.

PB - Springer-Verlag

T2 - 19th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2018

Y2 - 21 November 2018 through 23 November 2018

ER -

Vuttipittayamongkol P, Elyan E, Petrovski A, Jayne C. Overlap-Based Undersampling for Improving Imbalanced Data Classification. In Yin H, Novais P, Camacho D, Tallón-Ballesteros AJ, editors, Intelligent Data Engineering and Automated Learning – IDEAL 2018 - 19th International Conference, Proceedings. Springer-Verlag. 2018. p. 689-697. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-03493-1_72

Overlap-Based Undersampling for Improving Imbalanced Data Classification

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this