Feature selection based on multi-perspective entropy of mixing uncertainty measure in variable-granularity rough set | Applied Intelligence Skip to main content
Springer Nature Link
Log in

Feature selection based on multi-perspective entropy of mixing uncertainty measure in variable-granularity rough set

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Neighborhood rough set is an important model in feature selection. However, it only determines the granularity of the neighborhood from a feature perspective, while ignoring the influence of sample distribution on the granularity of the neighborhood. Moreover, the use of single and monotonic uncertainty measures limits its ability to obtain high-quality features in complex and large-scale data, as well as perform feature selection for high-dimensional data. To address these issues, we propose several improvements. Firstly, we construct a sample space state function to evaluate the influence of sample distribution on the granularity of the samples. Based on the state of the sample space, we then propose four perspectives of variable-granularity neighborhoods and define the upper and lower approximations for these neighborhoods. Additionally, we define the dependence of variable-granularity neighborhoods and the positive regions from an algebraic perspective. These definitions together form a comprehensive variable-granularity rough set model that can automatically adjust the granularity size according to the sample distribution and select high-quality features. Secondly, in order to comprehensively assess the uncertainty of data, we introduce the information and algebraic perspectives. We propose a multi-perspective mixed entropy measure in the variable-granularity rough set framework. Thirdly, to avoid the problem of selecting redundant features caused by using a monotonic evaluation function in classical neighborhood rough set feature selection algorithms, we propose a non-monotonic feature selection algorithm based on the individual perspective of samples and the mixed entropy measures of the variable-granularity rough set. Finally, to overcome the high time complexity of feature selection on high-dimensional data, we introduce Fish-Score as a preliminary dimensionality reduction technique. Experimental results on eleven datasets and using fifteen algorithms demonstrate that our method not only improves classification accuracy but also effectively reduces the number of features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of Data and Material

The UCI datasets used in the experiment are down-loaded from http://archive.ics.uci.edu/ml/index.php, the DNA microarray datasets used in the experiment are down-loaded from http://csse.szu.edu.cn/staff/zhuzx/.

References

  1. Zheng W, Chen S, Fu Z, Zhu F, Yan H, Yang J (2022) Feature selection boosted by unselected features. IEEE Transactions on Neural Networks and Learning Systems 33(9):4562–4574. https://doi.org/10.1109/TNNLS.2021.3058172

    Article  MathSciNet  Google Scholar 

  2. Wang P, Xue B, Liang J, Zhang M (2023) Feature clustering-Assisted feature selection with differential evolution. Pattern Recogn 140:109523. https://doi.org/10.1016/j.patcog.2023.109523

    Article  Google Scholar 

  3. Yang Y, Chen D, Zhang X, Ji Z, Zhang Y (2022) Incremental feature selection by sample selection and feature-based accelerator. Appl Soft Comput 121:108800. https://doi.org/10.1016/j.asoc.2022.108800

    Article  Google Scholar 

  4. Xu W, Huang M, Jiang Z, Qian Y (2023) Graph-based unsupervised feature selection for interval-valued information system. IEEE Trans Neural Netw Learn Syst 114. https://doi.org/10.1109/TNNLS.2023.3263684

  5. You D, Sun M, Liang S, Li R, Wang Y, Xiao J, Yuan F, Shen L, Wu X (2022) Online feature selection for multi-source streaming features. Inf Sci 590:267–295. https://doi.org/10.1016/j.ins.2022.01.008

    Article  Google Scholar 

  6. Zhou P, Wang N, Zhao S (2021) Online group streaming feature selection considering feature interaction. Knowl-Based Syst 226:107157. https://doi.org/10.1016/j.knosys.2021.107157

    Article  Google Scholar 

  7. Singh D, Singh B (2019) Hybridization of feature selection and feature weighting for high dimensional data. Appl Intell 49(4):1580–1596. https://doi.org/10.1007/s10489-018-1348-2

    Article  Google Scholar 

  8. Wang F, Liang J, Song P (2023) Coupling learning for feature selection in categorical data. Int J Mach Learn & Cyber 14:2455–2465. https://doi.org/10.1007/s13042-023-01775-z

    Article  Google Scholar 

  9. Peng L, Cai Z, Heidari AA, Zhang L, Chen H (2023) Hierarchical Harris hawks optimizer for feature selection. J Adv Res. https://doi.org/10.1016/j.jare.2023.01.014

    Article  Google Scholar 

  10. Wichitaksorn N, Kang Y, Zhang F (2023) Random feature selection using random subspace logistic regression. Expert Syst Appl 217:119535. https://doi.org/10.1016/j.eswa.2023.119535

    Article  Google Scholar 

  11. Xue Y, Zhu H, Neri F (2023) A feature selection approach based on NSGA-II with ReliefF. Appl Soft Comput 134:109987. https://doi.org/10.1016/j.asoc.2023.109987

    Article  Google Scholar 

  12. Dong L, Wang R, Chen D (2023) Incremental feature selection with fuzzy rough sets for dynamic data sets. Fuzzy Sets Syst 457:108503. https://doi.org/10.1016/j.fss.2023.03.006

    Article  MathSciNet  Google Scholar 

  13. Yang X, Zhang Y, Fujita H, Liu D, Li T (2020) Local temporal-spatial multi-granularity learning for sequential three-way granular computing. Inf Sci 541:75–97. https://doi.org/10.1016/j.ins.2020.06.020

    Article  MathSciNet  Google Scholar 

  14. Ju H, Ding W, Yang X, Fujita H, Xu S (2021) Robust supervised rough granular description model with the principle of justifiable granularity. Appl Soft Comput 110:107612. https://doi.org/10.1016/j.asoc.2021.107612

    Article  Google Scholar 

  15. Hu Q, Pan W, Zhang L, Zhang D, Song Y, Guo M, Yu D (2012) Feature selection for monotonic classification. IEEE Trans Fuzzy Syst 20(1):69–81. https://doi.org/10.1109/TFUZZ.2011.2167235

    Article  Google Scholar 

  16. Fujita H, Gaeta A, Loia V, Orciuoli F (2020) Hypotheses analysis and assessment in counterterrorism activities: A method based on owa and fuzzy probabilistic rough sets. IEEE Trans Fuzzy Syst 28(5):831–845. https://doi.org/10.1109/TFUZZ.2019.2955047

    Article  Google Scholar 

  17. Pawlak Z, Skowron A (2007) Rough sets: Some extensions. Inf Sci 177(1):28–40. https://doi.org/10.1016/j.ins.2006.06.006

    Article  MathSciNet  Google Scholar 

  18. Qian Y, Liang J, Yao Y, Dang C (2010) MGRS: A multi-granulation rough set. Inf Sci 180(6):949–970. https://doi.org/10.1016/j.ins.2009.11.023

    Article  MathSciNet  Google Scholar 

  19. She Y, He X (2012) On the structure of the multigranulation rough set model. Knowl-Based Syst 36:81–92. https://doi.org/10.1016/j.knosys.2012.05.019

    Article  Google Scholar 

  20. Peters J, Chan C-C, Grzymala-Busse JW, Ziarko W (2011) Preface: A rough set approach to data mining. Int J Intell Syst 26(6):497–498. https://doi.org/10.1002/int.20480

    Article  Google Scholar 

  21. Hu Q, Yu D, Liu J, Wu C (2008) Neighborhood rough set based heterogeneous feature subset selection. Inf Sci 178(18):3577–3594. https://doi.org/10.1016/j.ins.2008.05.024

    Article  MathSciNet  Google Scholar 

  22. Yong L, Wenliang H, Yunliang J, Zhiyong Z (2014) Quick attribute reduct algorithm for neighborhood rough set model. Inf Sci 271:65–81. https://doi.org/10.1016/j.ins.2014.02.093

    Article  MathSciNet  Google Scholar 

  23. Ping Y, Yongheng L (2011) Neighborhood rough set and SVM based hybrid credit scoring classifier. Expert Syst Appl 38(9):11300–11304. https://doi.org/10.1016/j.eswa.2011.02.179

    Article  Google Scholar 

  24. Chen Y, Zeng Z, Zhu Q, Tang C (2016) Three-way decision reduction in neighborhood systems. Appl Soft Comput 38:942–954. https://doi.org/10.1016/j.asoc.2015.10.059

    Article  Google Scholar 

  25. Pan Y, Xu W, Ran Q (2023) An incremental approach to feature selection using the weighted dominance-based neighborhood rough sets. Int J Mach Learn & Cyber 14(4):1217–1233. https://doi.org/10.1007/s13042-022-01695-4

    Article  Google Scholar 

  26. Li Y, Cai M, Zhou J, Li Q (2022) Accelerated multi-granularity reduction based on neighborhood rough sets. Appl Intell 52(15):17636–17651. https://doi.org/10.1007/s10489-022-03371-0

    Article  Google Scholar 

  27. Luo S, Miao D, Zhang Z, Zhang Y, Hu S (2020) A neighborhood rough set model with nominal metric embedding. Inf Sci 520:373–388. https://doi.org/10.1016/j.ins.2020.02.015

    Article  MathSciNet  Google Scholar 

  28. Hu M, Tsang ECC, Guo Y, Chen D, Xu W (2021) A novel approach to attribute reduction based on weighted neighborhood rough sets. Knowl-Based Syst 220:106908. https://doi.org/10.1016/j.knosys.2021.106908

    Article  Google Scholar 

  29. Xu J, Qu K, Sun Y, Yang J (2023) Feature selection using self-information uncertainty measures in neighborhood information systems. Appl Intell 53(4):4524–4540. https://doi.org/10.1007/s10489-022-03760-5

    Article  Google Scholar 

  30. Xu J, Meng X, Qu K, Sun Y, Hou Q (2023) Feature selection using relative dependency complement mutual information in fitting fuzzy rough set model. Appl Intell 53:18239–18262. https://doi.org/10.1007/s10489-022-04445-9

    Article  Google Scholar 

  31. Qu K, Xu J, Han Z, Xu S (2023) Maximum relevance minimum redundancy-based feature selection using rough mutual information in adaptive neighborhood rough sets. Appl Intell 53:17727–17746. https://doi.org/10.1007/s10489-022-04398-z

    Article  Google Scholar 

  32. Sun L, Yin T, Ding W, Qian Y, Xu J (2022) Feature selection with missing labels using multilabel fuzzy neighborhood rough sets and maximum relevance minimum redundancy. IEEE Trans Fuzzy Syst 30(5):1197–1211. https://doi.org/10.1109/TFUZZ.2021.3053844

    Article  Google Scholar 

  33. Chen Y, Xue Y, Ma Y, Xu F (2017) Measures of uncertainty for neighborhood rough sets. Knowl-Based Syst 120:226–235. https://doi.org/10.1016/j.knosys.2017.01.008

    Article  Google Scholar 

  34. Xie N, Liu M, Li Z, Zhang G (2019) New measures of uncertainty for an interval-valued information system. Inf Sci 470:156–174. https://doi.org/10.1016/j.ins.2018.08.047

    Article  MathSciNet  Google Scholar 

  35. Song Y, Zhang G, He J, Liao S, Xie N (2022) Uncertainty measurement for heterogeneous data: an application in attribute reduction. Artif Intell Rev 55(2):991–1027. https://doi.org/10.1007/s10462-021-09978-y

    Article  Google Scholar 

  36. Xu J, Sun Y, Qu K, Meng X, Hou Q (2022) Online group streaming feature selection using entropy-based uncertainty measures for fuzzy neighborhood rough sets. Complex Intell Syst 8(6):5309–5328. https://doi.org/10.1007/s40747-022-00763-0

    Article  Google Scholar 

  37. Sun L, Wang L, Ding W, Qian Y, Xu J (2021) Feature selection using fuzzy neighborhood entropy-based uncertainty measures for fuzzy neighborhood multigranulation rough sets. IEEE Trans Fuzzy Syst 29(1):19–33. https://doi.org/10.1109/TFUZZ.2020.2989098

    Article  Google Scholar 

  38. Xu J, Yuan M, Ma Y (2022) Feature selection using self-information and entropy-based uncertainty measure for fuzzy neighborhood rough set. Complex Intell Syst 8(1):287–305. https://doi.org/10.1007/s40747-021-00356-3

    Article  Google Scholar 

  39. Yang X, Chen H, Li T, Wan J, Sang B (2021) Neighborhood rough sets with distance metric learning for feature selection. Knowl-Based Syst 224:107076. https://doi.org/10.1016/j.knosys.2021.107076

    Article  Google Scholar 

  40. An S, Guo X, Wang C, Guo G, Dai J (2023) A soft neighborhood rough set model and its applications. Inf Sci 624:185–199. https://doi.org/10.1016/j.ins.2022.12.074

    Article  Google Scholar 

  41. Hu Q, Zhang L, Zhang D, Pan W, An S, Pedrycz W (2011) Measuring relevance between discrete and continuous features based on neighborhood mutual information. Expert Syst Appl 38(9):10737–10750. https://doi.org/10.1016/j.eswa.2011.01.023

    Article  Google Scholar 

  42. Chen Y, Wu K, Chen X, Tang C, Zhu Q (2014) An entropy-based uncertainty measurement approach in neighborhood systems. Inf Sci 279:239–250. https://doi.org/10.1016/j.ins.2014.03.117

    Article  MathSciNet  Google Scholar 

  43. Sun L, Zhang X-Y, Qian Y-H, Xu J-C, Zhang S-G, Tian Y (2019) Joint neighborhood entropy-based gene selection method with fisher score for tumor classification. Appl Intell 49(4):1245–1259. https://doi.org/10.1007/s10489-018-1320-1

    Article  Google Scholar 

  44. Zhang X, Fan Y, Yang J (2021) Feature selection based on fuzzy-neighborhood relative decision entropy. Pattern Recogn Lett 146:100–107. https://doi.org/10.1016/j.patrec.2021.03.001

    Article  Google Scholar 

  45. Qu K, Xu J, Hou Q, Qu K, Sun Y (2023) Feature selection using Information Gain and decision information in neighborhood decision system. Appl Soft Comput 136:110100. https://doi.org/10.1016/j.asoc.2023.110100

    Article  Google Scholar 

  46. Wang C, Shao M, He Q, Qian Y, Qi Y (2016) Feature subset selection based on fuzzy neighborhood rough sets. Knowl-Based Syst 111:173–179. https://doi.org/10.1016/j.knosys.2016.08.009

    Article  Google Scholar 

  47. Li S, Zhang K, Li Y, Wang S, Zhang S (2021) Online streaming feature selection based on neighborhood rough set. Appl Soft Comput 113:108025. https://doi.org/10.1016/j.asoc.2021.108025

    Article  Google Scholar 

  48. Gan M, Zhang L (2021) Iteratively local fisher score for feature selection. Appl Intell 51(8):6167–6181. https://doi.org/10.1007/s10489-020-02141-0

    Article  Google Scholar 

  49. Aran O, Akarun L (2010) A multi-class classification strategy for Fisher scores: Application to signer independent sign language recognition. Pattern Recogn 43(5):1776–1788. https://doi.org/10.1016/j.patcog.2009.12.002

    Article  Google Scholar 

  50. Chen H, Li T, Fan X, Luo C (2019) Feature selection for imbalanced data based on neighborhood rough sets. Inf Sci 483:1–20. https://doi.org/10.1016/j.ins.2019.01.041

    Article  Google Scholar 

  51. Liu J, Lin Y, Ding W, Zhang H, Wang C, Du J (2023) Multi-label feature selection based on label distribution and neighborhood rough set. Neurocomputing 524:142–157. https://doi.org/10.1016/j.neucom.2022.11.096

    Article  Google Scholar 

  52. Liu J, Lin Y, Li Y, Weng W, Wu S (2018) Online multi-label streaming feature selection based on neighborhood rough set. Pattern Recogn 84:273–287. https://doi.org/10.1016/j.patcog.2018.07.021

    Article  Google Scholar 

  53. Shu W, Qian W, Xie Y (2020) Incremental feature selection for dynamic hybrid data using neighborhood rough set. Knowl-Based Syst 194:105516. https://doi.org/10.1016/j.knosys.2020.105516

    Article  Google Scholar 

  54. Zou L, Ren S, Li H, Yang X (2021) An optimization of master S-N curve fitting method based on improved neighborhood rough set. IEEE Access 9:8404–8420. https://doi.org/10.1109/ACCESS.2021.3049403

    Article  Google Scholar 

  55. Zou L, Ren S, Sun Y, Yang X (2023) Attribute reduction algorithm of neighborhood rough set based on supervised granulation and its application. Soft Comput 27(3):1565–1582. https://doi.org/10.1007/s00500-022-07454-5

    Article  Google Scholar 

  56. Jensen R, Shen Q (2009) New approaches to fuzzy-rough feature selection. IEEE Trans Fuzzy Syst 17(4):824–838. https://doi.org/10.1109/TFUZZ.2008.924209

    Article  Google Scholar 

  57. Tan A, Wu W-Z, Qian Y, Liang J, Chen J, Li J (2019) Intuitionistic fuzzy rough set-based granular structures and attribute subset selection. IEEE Trans Fuzzy Syst 27(3):527–539. https://doi.org/10.1109/TFUZZ.2018.2862870

    Article  Google Scholar 

  58. Wang C, Huang Y, Shao M, Hu Q, Chen D (2020) Feature selection based on neighborhood self-information. IEEE Transactions on Cybernetics 50(9):4031–4042. https://doi.org/10.1109/TCYB.2019.2923430

    Article  Google Scholar 

  59. Qian Y, Wang Q, Cheng H, Liang J, Dang C (2015) Fuzzy-rough feature selection accelerator. Fuzzy Sets Syst 258:61–78. https://doi.org/10.1016/j.fss.2014.04.029

    Article  MathSciNet  Google Scholar 

  60. Chen D, Zhang L, Zhao S, Hu Q, Zhu P (2012) A novel algorithm for finding reducts with fuzzy rough sets. IEEE Trans Fuzzy Syst 20(2):385–389. https://doi.org/10.1109/TFUZZ.2011.2173695

    Article  Google Scholar 

  61. Sun L, Wang L, Ding W, Qian Y, Xu J (2020) Neighborhood multi-granulation rough sets-based attribute reduction using Lebesgue and entropy measures in incomplete neighborhood decision systems. Knowl-Based Syst 192:105373. https://doi.org/10.1016/j.knosys.2019.105373

    Article  Google Scholar 

  62. Xu J, Qu K, Meng X, Sun Y, Hou Q (2022) Feature selection based on multiview entropy measures in multiperspective rough set. Int J Intell Syst 37(10):7200–7234. https://doi.org/10.1002/int.22878

    Article  Google Scholar 

  63. Zeng K, She K, Niu X (2013) Multi-Granulation Entropy and Its Applications. Entropy 15(6):2288–2302. https://doi.org/10.3390/e15062288

    Article  MathSciNet  Google Scholar 

  64. Yu D, Hu Q, Wu C (2007) Uncertainty measures for fuzzy relations and their applications. Appl Soft Comput 7(3):1135–1143. https://doi.org/10.1016/j.asoc.2006.10.004

    Article  Google Scholar 

  65. Hu Q, Zhang L, Zhang D, Pan W, An S, Pedrycz W (2011) Measuring relevance between discrete and continuous features based on neighborhood mutual information. Expert Syst Appl 38(9):10737–10750. https://doi.org/10.1016/j.eswa.2011.01.023

    Article  Google Scholar 

  66. Sun L, Zhang X, Qian Y, Xu J, Zhang S (2019) Feature selection using neighborhood entropy-based uncertainty measures for gene expression data classification. Inf Sci 502:18–41. https://doi.org/10.1016/j.ins.2019.05.072

    Article  MathSciNet  Google Scholar 

  67. Chen Y, Zhang Z, Zheng J, Ma Y, Xue Y (2017) Gene selection for tumor classification using neighborhood rough sets and entropy measures. J Biomed Inform 67:59–68. https://doi.org/10.1016/j.jbi.2017.02.007

    Article  Google Scholar 

  68. Xu FF, Miao DQ, Wei L (2009) Fuzzy-rough attribute reduction via mutual information with an application to cancer classification. Computers & Mathematics with Applications 57(6):1010–1017. https://doi.org/10.1016/j.camwa.2008.10.027

    Article  Google Scholar 

  69. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Inst Stat Math 11(1):86–92

    Article  MathSciNet  Google Scholar 

  70. Dunn QJ (1961) Multiple comparisons among means. J Am Stat Assoc 56(293):52–64

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The paper is supported in part by the National Natural Science Foundation of China under Grant (61976082, 62076089, 62002103).

Author information

Authors and Affiliations

  1. College of Computer and Information Engineering, Henan Normal University, Xinxiang, 453007, China

    Jiucheng Xu, Changshun Zhou, Shihui Xu, Lei Zhang & Ziqin Han

  2. Engineering Lab of Intelligence Business & Internet of Things, Xinxiang, Henan Province, 453007, China

    Jiucheng Xu, Changshun Zhou, Shihui Xu, Lei Zhang & Ziqin Han

Authors
  1. Jiucheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changshun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shihui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ziqin Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Jiucheng Xu,Changshun Zhou; Methodology: Changshun Zhou; Writing - original draft preparation: Changshun Zhou, Shihui Xu, Lei Zhang; Writing - review and editing: Changshun Zhou, Ziqin Han; Funding acquisition: Jiucheng Xu.

Corresponding author

Correspondence to Changshun Zhou.

Ethics declarations

Competing interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A

Appendix A

Proof of property 1

when \({G_1} \subseteq {G_2}\), there is \(N_{KB}^{{G_1}}\left( L \right) \supseteq N_{KB}^{{G_2}} \left( L \right) \), from (28) and (30), we know \(\underline{N_{KB}^{{G_1}}} {E_z} \subseteq \underline{N_{KB}^{{G_2}}} {E_z}\) and \(\underline{N_{KB}^{{G_1}}} (E) \subseteq \underline{N_{KB}^{{G_2}}}( E)\), so \(\mid {\underline{N_{KB}^{{G_1}}} (E)} \mid \le \mid {\underline{N_{KB}^{{G_2}}}( E)} \mid \). From (31) and (32), \(\gamma _{KB}^{{G_1}}(E) \le \gamma _{KB}^{{G_2}}(E)\) holds.

Proof of property 2

Proof For \(\forall \ \ {L_i} \in U\),there is \(1 \le \mid {N_{KB}^B\left( {{L_i}} \right) } \mid \le \mid U \mid \), then \(\frac{1}{{\mid U \mid }} \le \frac{{\mid {N_{KB}^B\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} \le 1\), \(\log \frac{1}{{\mid U \mid }} \le \log \frac{{\mid {N_{KB}^B\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} \le 0\) and \(\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{1}{{\mid U \mid }} \le \mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^B\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} \le 0\), so \( \log \mid U \mid =- \log \frac{1}{{\mid U \mid }} \ge - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^B\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} \ge 0\), because \(0 \le \gamma _{KB}^B(E) \le 1\), so \(\log \mid U \mid \ge \gamma _{KB}^B(E)\log \mid U \mid = - \gamma _{KB}^B(E)\log \frac{1}{{\mid U \mid }} \ge - \frac{{\gamma _{KB}^B(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^B\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} \ge 0\). From (33), \(0 \le M{Q_{KB}}(B) \le \log \mid U \mid \) holds.

Proof of property 3

Proof From (33)–(34), we know

\(M{Q_{KB}}(b_2) = - \frac{{\gamma _{KB}^(b_2)(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^(b_2)\left( {{L_i}} \right) } \mid }}{{\mid U \mid }}\) and

\(M{Q_{KB}}({b_1}\mid {{b_2}} ) = - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\) then

\(M{Q_{KB}}({b_1}\mid {{b_2}} ) + M{Q_{KB}}({b_2})\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}}} - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\) \(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}}}} \right) \times \left( {\frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}} \right) \) \(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}\) \(= M{Q_{KB}}({b_1},{b_2})\), \(M{Q_{KB}}({b_1},{b_2}) = M{Q_{KB}}({b_1}\mid {{b_2}} ) + M{Q_{KB}}({b_2})\) holds.

Proof of property 4

Proof From (36), we know

\(M{Q_{KB}}({b_1};{b_2})\)

\(= - \frac{{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid U \mid }}\)

\(= - \frac{{\gamma _{KB}^{{b_2}}(E) + \gamma _{KB}^{{b_1}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) \cap N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid U \mid }}\)

\(= M{Q_{KB}}({b_2};{b_1})\), \(M{Q_{KB}}({b_1};{b_2}) = M{Q_{KB}}({b_2};{b_1})\) holds.

Proof of property 5

Proof From (33)–(34) and (36), we know

\(M{Q_{KB}}({b_1}) = - \frac{{\gamma _{KB}^{{b_1}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }}\),

\(M{Q_{KB}}({b_2}) = - \frac{{\gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }}\),

\(M{Q_{KB}}({b_1},{b_2}) = - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}\) and

\(M{Q_{KB}}({b_1};{b_2}) = - \frac{{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid U \mid }}\), then

\(M{Q_{KB}}({b_1}) + M{Q_{KB}}({b_2})\)

\(= - \frac{{\gamma _{KB}^{{b_1}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} - \frac{{\gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }}\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}}}\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \left( {\log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} + \log \frac{{{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} \times \frac{{{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}}}\),

\(M{Q_{KB}}({b_1}) + M{Q_{KB}}({b_2}) - M{Q_{KB}}({b_1},{b_2})\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \left( {\log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}}} - \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}}} \div \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}}} \times \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}}}{{\mid U{\mid ^{^{\gamma _{KB}^{{b_1}}(E)}{ + ^{\gamma _{KB}^{{b_2}}(E)}}}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\)

\(= - \frac{{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid U \mid }}\)

\(=M{Q_{KB}}({b_1};{b_2})\).

\(M{Q_{KB}}({b_1};{b_2}) = M{Q_{KB}}({b_1}) + M{Q_{KB}}({b_2}) - M{Q_{KB}}({b_1},{b_2})\) holds.

Proof of property 6

Proof From (33) and (35)–(36), we know

\(M{Q_{KB}}({b_1}) = - \frac{{\gamma _{KB}^{{b_1}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }}\),

\(M{Q_{KB}}({b_1}\mid {{b_2}} ) = - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\) and

\(M{Q_{KB}}({b_1};{b_2}) = - \frac{{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid U \mid }}\),

then \(M{Q_{KB}}({b_1}) - M{Q_{KB}}({b_1}\mid {{b_2}} )\)

\(= - \frac{{\gamma _{KB}^{{b_1}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }}{{\mid U \mid }} + \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} + \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \left( {\log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} - \log \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} \div \frac{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \left( {\frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E)}}}} \times \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}} \right) \)

\(= - \frac{1}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}{{\mid U{\mid ^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }^{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}}}\)

\(= - \frac{{\gamma _{KB}^{{b_1}}(E) + \gamma _{KB}^{{b_2}}(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) } \mid \mid {N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid }}{{\mid {N_{KB}^{{b_1}}\left( {{L_i}} \right) \cap N_{KB}^{{b_2}}\left( {{L_i}} \right) } \mid \mid U \mid }}\)

\(= M{Q_{KB}}({b_1};{b_2})\).

\(M{Q_{KB}}({b_1};{b_2}) = M{Q_{KB}}({b_1}) - M{Q_{KB}}({b_1}\mid {{b_2}} )\) holds.

Proof of property 7

Proof From Property 4 and 6, we know \(M{Q_{KB}}({b_1};{b_2}) \!= M{Q_{KB}}({b_2};{b_1})\) and \({Q_{KB}}({b_2};{b_1}) = M{Q_{KB}}({b_2}) - M{Q_{KB}}\) \(({b_2}\mid {{b_1}} \), then \(M{Q_{KB}}({b_1};{b_2}) = M{Q_{KB}}({b_2}) - M{Q_{KB}}({b_2}\mid {{b_1})}\) holds.

Proof of property 8

Proof From (38), we know

\(M{Q_{P{B^{pes}}}}(E;B) = - \frac{{1 + \gamma _{P{B^{pes}}}^B(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid }\log \) \( \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid \mid U \mid }}\) and

\(M{Q_{P{B^{pes}}}}(E;J)\mathrm{{ = }} - \frac{{1 + \gamma _{P{B^{pes}}}^J(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \) \( \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \mid U \mid }}\).

According (28)-(30) and \(J \subseteq B \subseteq C\), we know \(\mid {N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid \le \mid {N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \) and \(\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^B\left( {{L_i}} \right) }\) \( \mid \le \mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \), then the size between \(\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid \mid U \mid }}\) and \(\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \mid U \mid }}\) is unknown.

From Property 1, we know \(\frac{{1 + \gamma _{P{B^{pes}}}^J(E)}}{{\mid U \mid }} \le \frac{{1 + \gamma _{P{B^{pes}}}^B(E)}}{{\mid U \mid }}\), because the size between

\(\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid \mid U \mid }}\) and \(\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \mid U \mid }}\) is unknown, so the size of mutual information between

\(M{Q_{P{B^{pes}}}}(E;B) = - \frac{{1 + \gamma _{P{B^{pes}}}^B(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^B\left( {{L_i}} \right) } \mid \mid U \mid }}\) and

\(M{Q_{P{B^{pes}}}}(E;J)\mathrm{{ = }} - \frac{{1 + \gamma _{P{B^{pes}}}^J(E)}}{{\mid U \mid }}\mathop \sum \limits _{i = 1}^{\mid U \mid } \log \frac{{\mid {{N_E}\left( {{L_i}} \right) } \mid \mid {N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid }}{{\mid {{N_E}\left( {{L_i}} \right) \cap N_{P{B^{pes}}}^J\left( {{L_i}} \right) } \mid \mid U \mid }}\) is unknown, Property 8 holds.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Zhou, C., Xu, S. et al. Feature selection based on multi-perspective entropy of mixing uncertainty measure in variable-granularity rough set. Appl Intell 54, 147–168 (2024). https://doi.org/10.1007/s10489-023-05194-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05194-z

Keywords

Search

Navigation