A new kernel fuzzy based feature extraction method using attraction points

Abstract

This paper aims at introducing a novel supervised feature extraction method to be used in small sample size situations. The proposed approach considers the class membership of samples and exploits a nonlinear mapping in order to extract the relevant features and to mitigate the Hughes phenomenon. The proposed objective function is composed of three different terms, namely, attraction function, repulsion function, and the between-feature scatter matrix, where the last term increases the difference between extracted features. Subsequently, the attraction function and the repulsion function are redefined by incorporating the membership degrees of samples. Finally, the proposed method is extended using the kernel trick to capture the inherent nonlinearity of the original data. To evaluate the accuracy of the proposed feature extraction method, four remote sensing images are used in our experiments. The experiments indicate that the proposed feature extraction method is anappropriate choice for classification of hyperspectral images using limited training samples.

Appendices

Appendix A

Considering ${\mathbf{A}} = 4{\mathbf{UPU}}^{T}$, ${\mathbf{B}} = 2\gamma {\mathbf{Q}}^{T} {\mathbf{Q}}$, and ${\mathbf{C}} = {\mathbf{UU}}^{T}$, and applying the vec operator to Eq. (7) yield:

$$ \text{vec} ({\mathbf{TA}}) + \text{vec} ({\mathbf{BTC}}) - \lambda \text{vec} ({\mathbf{T}}) = 0 $$

(17)

Substituting $\text{vec} ({\mathbf{TA}})$ with $\text{vec} ({\mathbf{I}}_{m \times m} {\mathbf{TA}})$, where ${\mathbf{I}}_{m \times m}$ is an m × m identity matrix, and using the equality $\text{vec} ({\mathbf{abc}}) = ({\mathbf{c}}^{T} \otimes {\mathbf{a}})\text{vec} ({\mathbf{b}})$ (Kathrin 2004), where $\otimes$ is the Kronecker product, one can rewrite Eq. (17) as:

$$ ({\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} )vec({\mathbf{T}}) + ({\mathbf{C}}^{T} \otimes {\mathbf{B}})vec({\mathbf{T}}) - \lambda vec({\mathbf{T}}) = 0 $$

(18)

which is equal to Eq. (8).

Appendix B

Considering Eq. (6), one should maximize the following equation under the normalization constraint:

$$ 2\text{tr} ({\mathbf{TUPU}}^{T} {\mathbf{T}}^{T} ) + \gamma \text{tr} ({\mathbf{U}}^{T} {\mathbf{T}}^{T} {\mathbf{Q}}^{T} {\mathbf{QTU}}\varvec{)} $$

(19)

Using the circular property of trace, one may write:

$$ 2\text{tr} ({\mathbf{T}}^{T} {\mathbf{TUPU}}^{T} ) + \gamma \text{tr} ({\mathbf{T}}^{T} {\mathbf{Q}}^{T} {\mathbf{QTUU}}^{T} \varvec{)} $$

(20)

Using the equality $\text{tr} ({\mathbf{a}}^{T} {\mathbf{b}}) = \text{vec} ({\mathbf{a}})^{T} \text{vec} ({\mathbf{b}})$ (Kathrin 2004), one can rewrite Eq. (20) as:

$$ 2\text{vec} ({\mathbf{T}})^{T} \text{vec} ({\mathbf{TUPU}}^{T} ) + \gamma \text{vec} ({\mathbf{T}})^{T} \text{vec} ({\mathbf{Q}}^{T} {\mathbf{QTUU}}^{T} ) $$

(21)

Using the equality $\text{vec} ({\mathbf{abc}}) = ({\mathbf{c}}^{T} \otimes {\mathbf{a}})\text{vec} ({\mathbf{b}})$ (Kathrin 2004) and defining ${\mathbf{A}} = 4{\mathbf{UPU}}^{T}$, ${\mathbf{B}} = 2\gamma {\mathbf{Q}}^{T} {\mathbf{Q}}$, and ${\mathbf{C}} = {\mathbf{UU}}^{T}$, Eq. (21) can be rewritten as:

$$ \begin{aligned} & \text{vec} ({\mathbf{T}})^{T} ({\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} )\text{vec} ({\mathbf{T}}) + \text{vec} ({\mathbf{T}})^{T} ({\mathbf{C}}^{T} \otimes {\mathbf{B}})\text{vec} ({\mathbf{T}}) \\ & \quad = \text{vec} ({\mathbf{T}})^{T} ({\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} + {\mathbf{C}}^{T} \otimes {\mathbf{B}})\text{vec} ({\mathbf{T}}) \\ \end{aligned} $$

(22)

Due to the fact that $\text{vec} ({\mathbf{T}})$ is the eigenvector of ${\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} + {\mathbf{C}}^{T} \otimes {\mathbf{B}}$ (see Eq. (8)), one can conclude:

$$ \text{vec} ({\mathbf{T}})^{T} ({\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} + {\mathbf{C}}^{T} \otimes {\mathbf{B}})\text{vec} ({\mathbf{T}}) = \lambda \text{vec} ({\mathbf{T}})^{T} \text{vec} ({\mathbf{T}}) $$

(23)

Using the equality $\text{tr} ({\mathbf{a}}^{T} {\mathbf{b}}) = \text{vec} ({\mathbf{a}})^{T} \text{vec} ({\mathbf{b}})$ and considering the normalization constraint i.e. $\varvec{tr(}{\mathbf{TT}}^{T} \varvec{) = 1}$, yield:

$$ \lambda \text{vec} ({\mathbf{T}})^{T} \text{vec} ({\mathbf{T}}) = \lambda \text{tr} ({\mathbf{TT}}^{T} ) = \lambda $$

(24)

So, the maximum of Eq. (19) is obtained if $\text{vec} ({\mathbf{T}})$ is the eigenvector corresponding to the largest eigenvalue of ${\mathbf{A}}^{T} \otimes {\mathbf{I}}_{m \times m} + {\mathbf{C}}^{T} \otimes {\mathbf{B}}$.