The Bayesian network classifiers (BNCs) learned from labeled training data are expected to generalize to fit unlabeled testing data based on the independent and identically distributed (i.i.d.) assumption, whereas the asymmetric independence assertion demonstrates the uncertainty of significance of dependency or independency relationships mined from data. A highly scalable BNC should form a distinct decision boundary that can be especially tailored to specific testing instance for knowledge representation. To address the issue of asymmetric independence assertion, in this paper we propose to learn k-dependence Bayesian multinet classifiers in the framework of multistage classification. By partitioning training set and pseudo training set according to high-confidence class labels, the dependency or independency relationships can be fully mined and represented in the topologies of the committee members. Extensive experimental results indicate that the proposed algorithm achieves competitive classification performance compared to single-topology BNCs (e.g., CFWNB, AIWNB and SKDB) and ensemble BNCs (e.g., WATAN, SA2DE, ATODE and SLB) in terms of zero-one loss, root mean square error (RMSE), Friedman test and Nemenyi test.
AcidS.CamposL. and CastellanoJ.G., Learning Bayesian network classifiers: searching in a space of partially directed acyclic graphs, Machine Learning59(3) (2005), 213–235.
2.
KesavarajG. and SukumaranS., A study on classification techniques in data mining. In: Proceedings of the 4th International Conference on Computing, Communications and Networking Technologies, 2013, pp. 1–7.
3.
ScanagattaM.SalmerónA. and StellaF., A survey on Bayesian network structure learning from data, Progress in Artificial Intelligence8(4) (2019), 425–439.
4.
LiuY.WangL.M.MammadovM.ChenS.L.WangG.J.QiS.K. and SunM.H., Hierarchical independence thresholding for learning Bayesian network classifiers, Knowledge-Based Systems212 (2021), 106627.
5.
FriedmanN.GeigerD. and GoldszmidtM., Bayesian network classifiers, Machine Learning29(2-3) (1997), 131–163.
6.
BartlettM. and CussensJ., Integer linear programming for the Bayesian network structure learning problem, Artificial Intelligence244 (2017), 258–271.
7.
TillmanR.E., Structure learning with independent non-identically distributed data. In: Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 1041–1048.
8.
GanizM.C.GeorgeC. and PottengerW.M., Higher order naive Bayes: a novel non-IID approach to text classification, IEEE Transactions on Knowledge and Data Engineering23(7) (2010), 1022–1034.
9.
RyabkoD. and BartlettP., Pattern recognition for conditionally independent data, Journal of Machine Learning Research7 (2006), 645–664.
10.
GetoorL. and DiehlC.P., Link mining: a survey, ACM SIGKDD Explorations Newsletter7(2) (2005), 3–12.
11.
LiuY.WangL.M. and MammadovM., Learning semi-lazy Bayesian network classifier under the c.i.i.d assumption, Knowledge-Based Systems 208, 2020.
12.
BreimanL., Random forests, Machine Learning45(1) (2001), 5–32.
13.
WangL.M.XieY.B.PangM. and WeiJ.Y., Alleviating the attribute conditional independence and IID assumptions of averaged one-dependence estimator by double weighting, Knowledge-Based Systems250 (2022), 109078.
14.
JiangL.X.ZhangH.CaiZ.H. and WangD.H., Weighted average of one-dependence estimators, Journal of Experimental & Theoretical Artificial Intelligence24(2) (2012), 219–230.
15.
FreundY. and SchapireR.E., A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences55(1) (1997), 119–139.
16.
SagiO. and RokachL., Ensemble learning: a survey, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery8(4) (2018), e1249.
17.
SunY.WangL.M. and SunM.H., Label-driven learning framework: towards more accurate Bayesian network classifiers through discrimination of high-confidence labels, Entropy19(12) (2017), 661.
18.
LibalU. and HasiewiczZ., Risk upper bound for a NM-type multiresolution classification scheme of random signals by Daubechies wavelets, Engineering Applications of Artificial Intelligence62 (2017), 109–123.
19.
FienbergS.E. and KimS.H., Combining conditional log-linear structures, Journal of the American Statistical Association94(455) (1999), 229–239.
20.
KimG.H. and KimS.H., Marginal Information for Structure Learning, Statistics and Computing30(2) (2020), 331–349.
21.
LangleyP.IbaW. and ThompsonK., An analysis of bayesian classifiers, In: Proceedings of the 10th National Conference on Artificial Intelligence, 1992, pp. 223–228.
22.
JiangL.X.LiC.Q.WangS.S. and ZhangL.G., Deep feature weighting for naive Bayes and its application to text classification, Engineering Applications of Artificial Intelligence52 (2016), 26–39.
23.
RenY.WangL.M.LiX.F.PangM. and WeiJ.Y., Stochastic optimization for bayesian network classifiers, Applied Intelligence52(13) (2022), 15496–15516.
JiangL.X.CaiZ.H.WangD.H. and ZhangH., Improving tree augmented naive Bayes for class probability estimation, Knowledge-Based Systems26 (2012), 239–245.
26.
ShannonC.E., A mathematical theory of communication, The Bell System Technical Journal27(3) (1948), 379–423.
27.
MartinezA.M.WebbG.I.ChenS.L. and ZaidiN.A., Scalable learning of Bayesian network classifiers, Journal of Machine Learning Research17(1) (2016), 1515–1549.
28.
SahamiM., Learning limited dependence Bayesian classifiers, In: Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, 1996, pp 335–338.
29.
BielzaC. and LarranagaP., Discrete Bayesian network classifiers: A survey, ACM Computing Surveys47(1) (2014), 1–43.
30.
MurphyP.M. and AhaD.W., UCI Repository of Machine Learning Databases, Available online: http://www.ics.uci.edu/mlearn/MLRepository.html.
BelkasimS.ShridharM. and AhmadiM., Pattern classification using an efficient KNNR, Pattern Recognition25(10) (1992), 1269–1274.
33.
TsymbalA.PechenizkiyM.CunninghamP. and PuuronenS., Dynamic integration of classifiers for handling concept drift, Information Fusion9(1) (2008), 56–68.
34.
JiangL.X. and ZhangH., Lazy averaged one-dependence estimators, In: Proceedings of the 19th Conference of the Canadian Society for Computational Studies of Intelligence, 2006, pp. 515–525.
35.
DuanZ.Y.WangL.M.ChenS.L. and SunM.H., Instance-based weighting filter for superparent one-dependence estimators, Knowledge-Based Systems203 (2020), 106085.
36.
ZhangH.JiangL.X. and YuL.J., Attribute and instance weighted naive Bayes, Pattern Recognition111 (2021), 107674.
37.
MorrisonD.WangR.L.XuW.L. and SilvaL.D., Incremental learning for spoken affect classification and its application in call-centres, International Journal of Intelligent Systems Technologies and Applications2(2–3) (2007), 242–254.
38.
AlbornozE.M.MiloneD.H. and RufinerH.L., Spoken emotion recognition using hierarchical classifiers, Computer Speech & Language25(3) (2011), 556–570.
39.
SillaC.N. and FreitasA.A., A survey of hierarchical classification across different application domains, Data Mining and Knowledge Discovery22(1) (2011), 31–72.
40.
GrossiG.LanzarottiR. and LinJ.Y., Robust face recognition providing the identity and its reliability degree combining sparse representation and multiple features, International Journal of Pattern Recognition and Artificial Intelligence30(10) (2016), 1656007.
41.
LiuK.H.YanS.C. and KuoC.J., Age estimation via grouping and decision fusion, IEEE Transactions on Information Forensics and Security10(11) (2015), 2408–2423.
42.
BasuS.ChaudhuriC.KunduM.NasipuriM. and BasuD.K., A two-pass approach to pattern classification, In: Proceedings of the 11th International Conference on Neural Information Processing, 2004, pp 781–786.
43.
PoornaS.S. and NairG.J., Multistage classification scheme to enhance speech emotion recognition, International Journal of Speech Technology22(2) (2019), 327–340.
44.
GeigerD. and HeckermanD., Knowledge representation and inference in similarity networks and Bayesian multinets, Artificial Intelligence82(1-2) (1996), 45–74.
45.
HuangK.Z.KingI. and LyuM.R., Discriminative training of Bayesian Chow-Liu multinet classifiers. In: Proceedings of the International Joint Conference on Neural Networks, 2003, pp. 484–488.
46.
ParkS.H. and FürnkranzJ., Efficient implementation of class-based decomposition schemes for Naïve Bayes, Machine Learning96(3) (2014), 295–309.
47.
DomingosP., A unified bias-variance decomposition for zero-one and squared loss, In: Proceedings of the 17th National Conference on Artificial Intelligence, 2000, pp. 564–569.
48.
HyndmanR.J. and KoehlerA.B., Another look at measures of forecast accuracy, International Journal of Forecasting22(4) (2006), 679–688.
49.
DemsarJ., Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research7(1) (2006), 1–30.
50.
GarciaS. and HerreraF., An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons, Journal of Machine Learning Research9(12) (2008), 2677–2694.
51.
JiangL.X.ZhangL.G.LiC.Q. and WuJ., A correlation-based feature weighting filter for naive Bayes, IEEE Transactions on Knowledge and Data Engineering31(2) (2019), 201–213.
52.
ChenS.L.MartinezA.M.WebbG.I. and WangL.M., Selective AnDE for large data learning: a low-bias memory constrained approach, Knowledge and Information Systems50(2) (2017), 475–503.
53.
FayyadU. and IraniK., Multi-interval discretization of continuous-valued attributes for classification learning, In: Proceedings of the 13th International Joint Conference on Artificial Intelligence, 1993, pp. 1022–1029.
