Sage Journals: Discover world-class research

Abstract

Feature selection is a common solution to microarray analysis. Previous approaches either select features based on classical statistical tests that can be tuned up with a classifier, or using regularization penalties incorporated in the cost function. Here we propose to use a feature ranking and weighting scheme instead, which combines statistical techniques with a weighted $k$ -NN classifier using a modified forward selection procedure.

We demonstrate that classification accuracy of our proposal outperforms existing methods on a range of public microarray gene expression datasets. The proposed method is also compared to state-of-the-art feature selection algorithms by means of the Friedman test.

Although a bunch of feature selection techniques has been used for genomic data, the experimental results show the classification superiority of our method on most of the present gene expression datasets.

Keywords

Computational genomics microarray data analysis feature selection feature ranking feature weighting k-nearest neighbors

Get full access to this article

View all access options for this article.

References

Amaral

J.L.M.

Lopes

A.J.

Veiga

Faria

A.C.D.

and Melo

P.L.

, High-accuracy Detection of Airway Obstruction in Asthma Using Machine Learning Algorithms and Forced Oscillation Measurements, Comput Methods Programs Biomed (2017).

Apolloni

Leguizamón

and Alba

, Two hybrid wrapper-filter feature selection algorithms applied to high-dimensional microarray experiments, Appl Soft Comput 38 (2016), 922–932. doi: 10.1016/j.asoc.2015.10.037.

Biehl

Hammer

and Villmann

, Distance measures for prototype based classification, in: Int Work Brain-Inspired Comput, 2013, pp. 10.0–116.

Blum

A.L.

and Langley

, Selection of relevant features and examples in machine learning, Artif Intell 97 (1997), 245–271. doi: 10.1016/S0004-3702(97)00063-5.

Bolón-Canedo

Sánchez-Maroño

and Alonso-Betanzos

, An ensemble of filters and classifiers for microarray data classification, Pattern Recognit 45 (2012), 531–539. doi: 10.1016/j.patcog.2011.06.006.

Bolón-Canedo

Sánchez-Maroñp

and Alonso-Betanzos

, Distributed feature selection: An application to microarray data classification, Appl Soft Comput 30 (2015), 136–150. doi: 10.1016/j.asoc.2015.01.035.

Chan

W.H.

Mohamad

M.S.

Deris

Zaki

Kasim

Omatu

Corchado

J.M.

and Al Ashwal

, Identification of informative genes and pathways using an improved penalized support vector machine with a weighting scheme, Comput Biol Med 77 (2016), 102–115. doi: 10.1016/j.compbiomed.2016.08.004.

Chandrashekar

and Sahin

, A survey on feature selection methods, Comput Electr Eng 40 (2014), 16–28. doi: 10.1016/j.compeleceng.2013.11.024.

Clemmensen

Hastie

Witten

and Ersbøll

, Sparse Discriminant Analysis, Technometrics 53 (2011), 406–413. doi: 10.1198/TECH.2011.08118.

10.

Cover

and Hart

, Nearest neighbor pattern classification, IEEE Trans Inf Theory 13 (1967), 21–27. doi: 10.1109/TIT.1967.1053964.

11.

Demšar

, Statistical comparisons of classifiers over multiple data sets, J Mach Learn Res 7 (2006), 1–30. doi: 10.1.1.141.3142.

12.

Deng

Zhu

Cheng

Zong

and Zhang

, Efficient kNN classification algorithm for big data, Neurocomputing 195 (2016), 143–148.

13.

Elyasigomari

Lee

D.A.

Screen

H.R.C.

and Shaheed

M.H.

, Development of a two-stage gene selection method that incorporates a novel hybrid approach using the cuckoo optimization algorithm and harmony search for cancer classification, J Biomed Inform 67 (2017), 11–20.

14.

Ertuugrul

öF.

and Taugluk

M.E.

, A novel version of k nearest neighbor: Dependent nearest neighbor, Appl SoftComput 55 (2017), 480–490.

15.

Fix

and Hodges

J.L.

, Discriminatory analysis nonparametric discrimination: Consistency properties, Int Stat Rev/Rev Int Stat 57 (1989), 238. doi: 10.2307/1403797.

16.

Friedman

, The Use of Ranks to Avoid the Assumption of Normality Implicit in the analysis of variance, J Am Stat Assoc 32 (1937), 675–701. doi: 10.1080/01621459.1937.10503522.

17.

Gali

Mariescu-Istodor

and Fränti

, Using linguistic features to automatically extract web page title, Expert Syst Appl 79 (2017), 296–312.

18.

Golub

T.R.

, Molecular Classification of cancer: Class discovery and class prediction by gene expression monitoring, Science (80-.) 286 (1999), 531–537. doi: 10.1126/science.286.5439.531.

19.

Guyon

and Elisseeff

, An introduction to variable and feature selection, J Mach Learn Res 3 (2003), 1157–1182.

20.

Guyon

and Elisseeff

, An introduction to feature extraction, in: Featur Extr, Springer, 2006, pp. 1–25.

21.

Guyon

Weston

Barnhill

and Vapnik

, Gene selection for cancer classification using support vector machines, Mach Learn 46 (2002), 389–422.

22.

Hall

Frank

Holmes

Pfahringer

Reutemann

and Witten

I.H.

, The WEKA data mining software: an update, ACM SIGKDD Explor Newsl 11 (2009), 10–18.

23.

Hira

Z.M.

and Gillies

D.F.

, A review of feature selection and feature extraction methods applied on microarray data, Adv Bioinformatics 2015 (2015), 1–13. doi: 10.1155/2015/198363.

24.

Hsu

H.-H.

Hsieh

C.-W.

and Lu

M.-D.

, Hybrid feature selection by combining filters and wrappers, Expert Syst Appl 38 (2011), 8144–8150.

25.

James

Witten

Hastie

and Tibshirani

, An Introduction to Statistical Learning, Springer, 2013.

26.

Jeffery

I.B.

Higgins

D.G.

and Culhane

A.C.

, Comparison and evaluation of methods for generating differentially expressed gene lists from microarray data, BMC Bioinformatics 7 (2006), 359.

27.

Kira

and Rendell

L.A.

, The feature selection problem: Traditional methods and a new algorithm, in: AAAI, 1992, pp. 129–134.

28.

Kohavi

and John

G.H.

, Wrappers for feature subset selection, Artif Intell 97 (1997), 273–324. doi: 10.1016/S0004-3702(97)00043-X.

29.

Lee

C.P.

and Lin

W.S.

, Using the two-population genetic algorithm with distance-based k-nearest neighbour voting classifier for high-dimensional data, Int J Data Min Bioinform 14 (2016), 315. doi: 10.1504/IJDMB.2016.075820.

30.

Cheng

Wang

Morstatter

Trevino

R.P.

Tang

and Liu

, Feature selection: A data perspective, ACM Comput Surv 50 (2017), 94.

31.

Chen

Yan

Jin

Xue

and Gao

, A hybrid feature selection algorithm for gene expression data classification, Neurocomputing (2017).

32.

Maillo

Ramírez

Triguero

and Herrera

, kNN-IS: An Iterative Spark-based design of the k-Nearest Neighbors classifier for big data, Knowledge-Based Syst. 117 (2017), 3–15.

33.

Mandal

and Mukhopadhyay

, Multiobjective PSO-based rank aggregation: Application in gene ranking from microarray data, Inf Sci (Ny) 385 (2017), 55–75.

34.

Molina

L.C.

Belanche

and Nebot

, Feature selection algorithms: A survey and experimental evaluation, in: 2002 IEEE Int. Conf. Data Mining, 2002. Proceedings, IEEE Comput. Soc, 2002, pp. 306–313. doi: 10.1109/ICDM.2002.1183917.

35.

Mortazavi

and Moattar

M.H.

, Robust feature selection from microarray data based on cooperative game theory and qualitative mutual information, Adv Bioinformatics 2016 (2016), 1–16. doi: 10.1155/2016/1058305.

36.

Narendra

P.M.

and Fukunaga

, A branch and bound algorithm for feature subset selection, IEEE Trans Comput 26 (1977), 917–922. doi: 10.1109/tc.1977.1674939.

37.

Nemenyi

P.B.

, Distribution-free multiple comparisons, Princeton University, 1963.

38.

Niu

and Wang

, Bacterial Colony Optimization, Discret Dyn Nat Soc 2012 (2012), 1–28. doi: 10.1155/2012/698057.

39.

Pan

Wang

and Ku

, A new general nearest neighbor classification based on the mutual neighborhood information, Knowledge-Based Syst. 121 (2017), 142–152.

40.

Park

C.H.

and Kim

S.B.

, Sequential random k-nearest neighbor feature selection for high-dimensional data, Expert Syst. Appl 42 (2015), 2336–2342. doi: 10.1016/j.eswa.2014.10.044.

41.

Shreem

S.S.

Abdullah

and Nazri

M.Z.A.

, Hybrid feature selection algorithm using symmetrical uncertainty and a harmony search algorithm, Int J Syst Sci 47 (2016), 1312–1329.

42.

Song

Liang

and Zhao

, An efficient instance selection algorithm for k nearest neighbor regression, Neurocomputing (2017).

43.

Sun

Wong

A.K.C.

and Kamel

M.S.

, Classification of imbalanced data: A review, Int J Pattern Recognit Artif Intell 23 (2009), 687–719.

44.

Tang

Alelyani

and Liu

, Feature selection for classification: A review, Data Classif Algorithms Appl (2014), 37.

45.

Tibshirani

, Regression shrinkage and selection via the lasso, J R Stat Soc Ser B (1996), 267–288.

46.

Trstenjak

Mikac

and Donko

, KNN with TF-IDF based framework for text categorization, Procedia Eng 69 (2014), 1356–1364.

47.

Walters-Williams

and Li

, Comparative study of distance functions for nearest neighbors, Adv Tech Comput Sci Softw Eng (2010), 79–84.

48.

Wang

Chen

and Alterovitz

, Improving PLS-RFE based gene selection for microarray data classification, Comput Biol Med 62 (2015), 14–24. doi: 10.1016/j.compbiomed.2015.04.011.

49.

Wang

Chen

and Alterovitz

, Accelerating wrapper-based feature selection with K-nearest-neighbor, Knowledge-Based Syst 83 (2015), 81–91. doi: 10.1016/j.knosys.2015.03.009.

50.

Wang

Jing

and Niu

, A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data, Knowledge-Based Syst 126 (2017), 8–19. doi: 10.1016/j.knosys.2017.04.004.

51.

Wei

Wan

Guo

and Wong

K.K.L.

, A novel hierarchical selective ensemble classifier with bioinformatics application, Artif Intell Med (2017).

52.

Weinberger

K.Q.

and Saul

L.K.

, Distance metric learning for large margin nearest neighbor classification, J Mach Learn Res 10 (2009), 207–244.

53.

Wettschereck

Aha

D.W.

and Mohri

, A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms, Artif Intell Rev 11 (1997), 273–314.

54.

Yang

C.-H.

Weng

Z.-J.

Chuang

L.-Y.

and Yang

C.-S.

, Identification of SNP-SNP interaction for chronic dialysis patients, Comput Biol Med 83 (2017), 94–101.

55.

Yang

C.-H.

Chuang

L.-Y.

and Yang

C.H.

, IG-GA: a hybrid filter/wrapper method for feature selection of microarray data, J Med Biol Eng 30 (2010), 23–28.

56.

Yang

Zhou

Zhu

and Ji

, Iterative ensemble feature selection for multiclass classification of imbalanced microarray data, J Biol Res 23 (2016), 13.

57.

Zhang

and Zhang

, Significance of gene ranking for classification of microarray samples, IEEE/ACM Trans Comput Biol Bioinforma 3 (2006), 312–320. doi: 10.1109/TCBB.2006.42.

58.

Zhang

Kotagiri

Tari

and Cheriet

, KRNN: k Rare-class Nearest Neighbour classification, Pattern Recognit 62 (2017), 33–44.

59.

Zou

and Hastie

, Regularization and variable selection via the elastic net, J R Stat Soc Ser B 67 (2005), 301–320. doi: 10.1111/j.1467-9868.2005.00503.x.

60.

Cancer microarray data sets. Plymouth University, (2005). http://www.tech.plym.ac.uk/spmc/links/bioinformatics/microarray/microarray_cancers.html (accessed February 14, 2017).