An Enhanced Affine Projection Algorithm Based on the Adjustment of Input-Vector Number

Abstract

An enhanced affine projection algorithm (APA) is proposed to improve the filter performance in aspects of convergence rate and steady-state estimation error, since the adjustment of the input-vector number can be an effective way to increase the convergence rate and to decrease the steady-state estimation error at the same time. In this proposed algorithm, the input-vector number of APA is adjusted reasonably at every iteration by comparing the averages of the accumulated squared errors. Although the conventional APA has the constraint that the input-vector number should be integer, the proposed APA relaxes that integer-constraint through a pseudo-fractional method. Since the input-vector number can be updated at every iteration more precisely based on the pseudo-fractional method, the filter performance of the proposed APA can be improved. According to our simulation results, it is demonstrated that the proposed APA has a smaller steady-state estimation error compared to the existing APA-type filters in various scenarios.

1. Introduction

Many representative algorithms based on the adaptive filtering theory have been successfully applied in channel estimation, system identification, and noise/echo cancellation [1,2,3,4,5,6,7,8,9,10,11,12,13]. As can be seen in Figure 1, the main purpose of an adaptive filter is the accomplishment for the precise estimation of filter coefficients, to minimize error signals with the same input signals. The least-mean squares (LMS) and normalized least-mean squares (NLMS) algorithms are the representative adaptive filtering algorithms because of easy implementation and low complexity. Compared to the LMS and NLMS algorithms [9], the affine projection algorithm (APA) [7] exhibits a high convergence rate for highly correlated input data because it employs multiple input vectors instead of only one input vector. However, the APA is computationally complex and incurs a large steady-state estimation error. Notably, a high input-vector number leads to rapid convergence but a large estimation error. In contrast, a low input-vector number leads to slow convergence but a small estimation error. Therefore, by adjusting the input-vector number, a high convergence rate and small steady-state estimation error can likely be simultaneously realized.

Recently, several researchers have focused on the input-vector number to enhance the performance of APAs, with several representative algorithms being the APA with dynamic selection of input vectors (DS-APA) [14], APA with selective regressors (SR-APA) [15], and APA with evolving order (E-APA) [16]. Although these algorithms outperform the conventional APA by achieving smaller estimation errors, the convergence rate and steady-state estimation error remain to be optimized.

Considering these aspects, in this study, a novel variable input-vector number APA framework is developed, in which the input-vector number is modified using a pseudo-fractional method based on the concept of the pseudo-fractional tap length [17] to ensure small steady-state estimation errors. The pseudo-fractional method employs the integer and fractional input-vector number and relaxes the constraint of the conventional APA that requires the input-vector number to be integer. Thus, the input-vector number for the proposed APA can be increased or decreased by comparing the averages of the accumulated errors. Compared to the existing algorithms, such as the conventional APA, DS-APA, SR-APA, and E-APA, the proposed algorithm based on the pseudo-fractional method achieves a higher convergence rate and smaller steady-state estimation error.

This paper is organized as follows. In the following Section 2, the conventional APA is explained. In Section 3, the proposed APA based on the adjustment of input-vector number by using the pseudo-fractional strategy is explained in detail. In Section 4, we present the simulation results to verify the performance of the proposed APA. Finally, Section 5 gives the conclusions of this paper.

2. Conventional Affine Projection Algorithm

Consider reference data $d_i$ obtained from an unknown system

$$\begin{array}{c}\hfill d_i={\mathbf{u}}_i^T{\mathbf{w}}_o+v_i,\end{array}$$

(1)

where ${\mathbf{w}}_o$ is the n-dimensional column vector of the unknown system, which must be estimated; $v_i$ indicates the measurement noise, with variance ${\sigma }_v^2$; and ${\mathbf{u}}_i$ denotes an n-dimensional column input vector, ${\mathbf{u}}_i={[u_i{u}_{i-1}\dots u_{i-n+1}]}^T$. The conventional APA is derived by minimizing the ${\mathcal{L}}_2$-norm of the difference of filter coefficient vectors between at iteration i and $i+1$ with setting a posteriori the error vector as zero, as follows:

$$\underset{{\widehat{\mathbf{w}}}_{i+1}}{min}\left|\right|{\widehat{\mathbf{w}}}_{i+1}-{\widehat{\mathbf{w}}}_i{\left|\right|}_2^2\mathrm{subject}\mathrm{to}{\mathbf{d}}_i={\mathbf{U}}_i^T{\widehat{\mathbf{w}}}_{i+1},$$

(2)

where ${\mathbf{e}}_i={\mathbf{d}}_i-{\mathbf{U}}_i^T{\widehat{\mathbf{w}}}_i$, ${\widehat{\mathbf{w}}}_i$ is the estimate of ${\mathbf{w}}_o$ at iteration i, $\mu$ is the step-size parameter, M is the input-vector number that means the number of current input vectors used for the APA update, and

$$\begin{array}{c}\hfill {\mathbf{U}}_i=[{\mathbf{u}}_i{\mathbf{u}}_{i-1}\dots {\mathbf{u}}_{i-M+1}],{\mathbf{d}}_i={[d_i{d}_{i-1}\dots d_{i-M+1}]}^T.\end{array}$$

(3)

Through the Lagrange multiplier and gradient descent method, the filter update equation of the conventional APA can be presented as [7]

$$\begin{array}{c}\hfill {\widehat{\mathbf{w}}}_{i+1}={\widehat{\mathbf{w}}}_i+\mu {\mathbf{U}}_i{({\mathbf{U}}_i^T{\mathbf{U}}_i+\beta \mathbf{I})}^{-1}{\mathbf{e}}_i,\end{array}$$

(4)

where $\beta$ is the regularization parameter that is usually a very small positive value, and $\mathbf{I}$ denotes the identity matrix.

3. Enhanced Affine Projection Algorithm Based on the Adjustment of Input-Vector Number

The conventional APA has the constraint that the input-vector number of APA should be integer. In this context, the filter performance can be enhanced if the input-vector number can include both integer and non-integer parts. Therefore, we develop a novel APA by the adjustment of the input-vector number using a pseudo-fractional method that is based on the concept of the pseudo-fractional tap length [17]. The pseudo-fractional method adopts the integer and fractional input-vector number by relaxing the integer constraint for the input-vector number. The integer input-vector number refers to the integer part of the fractional input-vector number when the difference in the integer and fractional input-vector number is larger than a predetermined value. In this proposed framework, the input-vector numbers are dynamically adjusted to enhance the performance of the proposed algorithm in terms of the convergence rate and steady-state estimation error. Moreover, the leaky factor is implemented in the adaptation rule of the fractional input-vector number.

According to the adaptation rule, as defined in Equation (5), the integer input-vector number remains constant until the change in the fractional input-vector number accumulates to a certain extent. $P_i$ is the pseudo-fractional input-vector number, which can be assigned positive integer values.

$$\begin{array}{c}\hfill P_{i+1}=\left\{\begin{array}{cc}\hfill & (P_i-\alpha )-\gamma (AAS{E}_{M_i}\left(i\right)-AAS{E}_{M_i-1}\left(i\right)),\mathrm{if}{M}_i⩾2\hfill \\ \hfill & (P_i-\alpha )-\gamma (AAS{E}_{M_i+1}\left(i\right)-AAS{E}_{M_i}\left(i\right)),\mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(5)

In this expression, $\alpha$ and $\gamma$ are small positive numbers, $\alpha$ is the leaky factor that satisfies $\alpha \ll \gamma$, and $M_i$ is the integer input-vector number at instant i. AASE, which is average of the accumulated squared errors, is defined as

$$\begin{array}{c}\hfill AASE{}_M\left(i\right)\triangleq \frac{{\sum }_{N=0}^{M-1}{e}_N^2\left(i\right)}{M}.\end{array}$$

(6)

Subsequently, the integer input-vector number $M_i$ can be determined as

$$\begin{array}{c}\hfill M_i=\left\{\begin{array}{cc}\hfill & max\{min\{\lfloor P_{i-1}\rfloor ,M_{max}\},1\},\mathrm{if}|M_{i-1}-P_{i-1}|⩾\delta \hfill \\ \hfill & M_{i-1},\mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(7)

where the $\lfloor .\rfloor$ operator approximates the value to the nearest integer, and $\delta$ is the threshold parameter. $M_i$ is updated to satisfy $1\le M_i\le M_{max}$, where $M_{max}$ is the maximum input-vector number. The threshold parameter $\delta$ is set to 0.5 to maximize the filter performance of our proposed APA. The proposed APA is updated using the following expression:

$$\begin{array}{c}\hfill {\widehat{\mathbf{w}}}_{i+1}={\widehat{\mathbf{w}}}_i+\mu {\mathbf{U}}_{i,M_i}{({\mathbf{U}}_{i,M_i}^T{\mathbf{U}}_{i,M_i}+\beta \mathbf{I})}^{-1}{\mathbf{e}}_{i,M_i}.\end{array}$$

(8)

where ${\mathbf{U}}_{i,M_i}=[{\mathbf{u}}_i{\mathbf{u}}_{i-1}\dots {\mathbf{u}}_{i-M_i+1}]$, ${\mathbf{e}}_{i,M_i}={[e_0\left(i\right)e_1\left(i\right)\dots e_{M_i-1}\left(i\right)]}^T$, and $M_i$ is determined according to the adaptation rule for the fractional input-vector number. Through the above-mentioned method, the proposed APA can accomplish the variable input-vector number strategy to enhance the filter performance in terms of the convergence rate and steady-state estimation error.

Because the proposed algorithm is designed for stationary environments, the input-vector number must be re-initialized to achieve a high tracking performance each time the target system is changed. To this end, the re-initialization method reported in [18] is used as a reference and modified, as illustrated in Algorithm 1.

Algorithm 1: Re-initialization of the input-vector number.

$e_{th}\triangleq \mu {\sigma }_v^2{M}_{max}/(2-\mu )$, flag $=0$, $e_{avg}=e_0^2$,

$\lambda ,{\alpha }_1,{\alpha }_2$: user defined.

for each i do

if ($e_i^2<{\alpha }_1\ast e_{th}$)

flag $=1$

else if ( flag = 1 and ${\alpha }_2\ast e_{avg}<e_i^2$ )

flag $=0$, $e_{avg}=e_i^2$, $M_i=M_{max}$, $P_i=M_{max}$

end if

$e_{avg}=\lambda e_{avg}+(1-\lambda )e_i^2$

end for

4. Experimental Results

The performance of the proposed algorithm is evaluated considering a channel estimation framework. The channel of the unknown system is generated using a moving average model with 16 taps ($n=16$). The adaptive filter and unknown channel are assumed to have the same number of taps. Moreover, the noise variance ${{\sigma }_v}^2$ is assumed to be known a priori, as its value can be estimated during silences in many practical applications [19,20,21]. The input signal $u_i$ is generated by filtering a white, zero-mean Gaussian random sequence by using the following system:

$$\begin{array}{ccc}\hfill G_1\left(z\right)& =\frac{1}{1-0.9z^{-1}},G_2\left(z\right)\hfill & \hfill =\frac{1+0.6z^{-1}}{1+z^{-1}+0.21z^{-2}}.\end{array}$$

(9)

The measurement noise $v_i$ is added to $y_i$ with a signal-to-noise ratio (SNR) of 30 dB. In this case, the SNR is defined by $10{log}_{10}(E\left[y_i^2\right]/E\left[{\upsilon }_i^2\right])$ and $y_i={\mathbf{u}}_i^T{\mathbf{w}}_o$. $P_0$ and $M_0$ are set to $M_{max}$, which is the initial input-vector number of the proposed APA. The mean squared deviation (MSD), i.e., $E\parallel {\mathbf{w}}_o-{\widehat{\mathbf{w}}}_i{\parallel }^2$, is adopted as the indicator for the algorithm performance. The simulation results are obtained through ensemble averaging over 100 independent trials, and the input signals are generated using $G_1\left(z\right)$ and $G_2\left(z\right)$. Furthermore, to examine the tracking performance of the proposed algorithm, the coefficients of the unknown filter taps are abruptly varied at time $5\times {10}^3$ in the simulation. The parameters of the proposed algorithm are set as follows: $M_{max}$ = 8, $\mu$ = 0.1, $\alpha$ = 0.03, $\gamma =1-\alpha$, and $\beta =0.000001$. The re-initialization parameters are set as $\lambda$ = 0.95, ${\alpha }_1$ = 10, ${\alpha }_2$ = 40 [18].

4.1. System Identification Verification for Correlated Input

Figure 2 shows the MSD learning curves of the conventional APA [7], DS-APA [14], SR-APA [15], E-APA [16], and proposed APA when the input vector is generated using $G_1\left(z\right)$ and the AR input model. As can be seen in Figure 2, the proposed APA has the smaller steady-state estimation error as compared to the existing APA-type algorithms. In addition, even though the system change occurs suddenly as shown in Figure 3, the proposed algorithm maintains the filter performance in aspects of the convergence rate and steady-state estimation error. Moreover, when the tap length is long ($n=256$), the proposed APA also has the smaller steady-state estimation errors, as shown in Figure 4. Figure 5 also shows the MSD learning curves of the conventional APA, DS-APA, SR-APA, E-APA, and proposed APA when the input vector is generated using $G_2\left(z\right)$ and the ARMA input model. As can be seen in Figure 5, the proposed APA outperforms the existing algorithm in terms of the convergence rate and steady-state estimation error. Figure 6 represents the comparison of input vector numbers over one trial for the E-APA and proposed-APA. According to Figure 6, it can be verified that the proposed APA uses the input-vector number efficiently. Figure 7 shows the tracking performance of the existing and proposed algorithms when the unknown system is abruptly changed from ${\mathbf{w}}_o$ to $-{\mathbf{w}}_o$. As can be seen in Figure 3 and Figure 7, the tracking performance of the proposed algorithm is maintained without any degradation in the convergence rate or steady-state estimation errors, owing to the use of the re-initialization method [18]. These simulation results demonstrate that the proposed variable input-vector number APA achieves a smaller steady-state estimation error than the existing algorithms.

In our proposed APA, it is important to decide on the values of $\alpha$ and $\gamma =1-\alpha$ in Equation (5), because the parameters $\alpha$ and $\gamma$ are dominant factors to determine the input-vector number. Therefore, we investigate the filter performance comparison according to several values of $\alpha$ and $\gamma$. As shown in Figure 8, the proposed APA has the best performance in case of $\alpha =0.03$ with $\gamma =1-\alpha$. Even though the parameter tuning method cannot precisely provide an optimal value, $\alpha =0.03$ gives the best performance in our simulation scenarios. Since the parameter $\alpha$ and $\gamma$ values have a restriction between 0 and 1, it is easy to find the proper $\alpha$ and $\gamma$ values based on the variation of $\alpha$ and $\gamma$ values to ensure the improved filter performance in each scenario.

Moreover, the parameter $\delta$ in Equation (7) is also a dominant factor to decide on the filter performance of the proposed APA. To verify the influence of the parameter $\delta$, the comparison simulation according to the variation of $\delta$ is shown in Figure 9. Since the filter performance of the proposed APA can be maximized experimentally when the parameter $\delta$ is set to 0.5, the proposed APA uses $\delta =0.5$ consistently in all simulations.

4.2. Speech Input Verification Including a Double-Talk Situation

The proposed APA was experimented by using the speech input signals to ensure the filter performance in practical scenarios as can be seen in Figure 10. Since the speech input signals are real human speech data, this simulation result for speech input increases the reliability of the proposed APA for practical use. As can be seen in Figure 11, we can find that the proposed APA can accomplish the smaller steady-state estimation errors compared to the other algorithms. The proposed algorithm was also tested in a double-talk situation as shown in Figure 12. The far-end input signal and near-end input signal were speech signals where the power of the near-end input signal was 2 times greater than that of the far-end input signal. The near-end input signal was added between iterations $6.2\times {10}^3$ and $7.2\times {10}^3$. Figure 12 shows that the proposed APSA delivered better performance than the other algorithms in terms of the convergence rate and the steady-state estimation error. Even after the double-talk occurrence, we can find that the proposed APSA consistently has smaller steady-state estimation errors. As can be seen in Figure 11 and Figure 12, the proposed APA can achieve the better filter performance in aspects of the convergence rate and steady-state estimation error in harsh speech-input scenarios.

4.3. Comparison for Computational Complexity

Table 1. Computational complexity of conventional APA, DS-APA, E-APA, and proposed APA.

	APA	DS-APA	E-APA	Proposed APA
Input-Vector
Number	M	$M_j$	$M_k$	$M_i$
$\#(\times /÷)$	$(M^2+2M)n$	$(M_j^2+M_j+M)n$	$(M_k^2+2M_k)n+$	$(M_i^2+2M_i)n+$
	$+M^3+M^2$	$+M_j^3+M_j^2$	$M_k^3+M_k^2$	$M_i^3+M_i^2$
	$+M^3+M^2$	$+M_j^3+M_j^2$	$+M_k+1$	$+M_i+2$
#(comparisons)	0	M	2	2

presents the iteration-wise computational complexity values of the conventional APA, DS-APA, E-APA, and proposed algorithm, for which the input-vector numbers are M, $M_j$, $M_k$, and $M_i$, respectively. Figure 13 shows the accumulated sum for the multiplications of the proposed APA compared to the existing APA-type algorithms. As can be seen in Figure 13, the proposed algorithm can be executed with smaller computational complexity than the other algorithms. The values for the proposed algorithm are considerably smaller than those of the conventional APA, DS-APA, and E-APA owing to the considerably smaller input-vector numbers of the proposed algorithm in the steady state.

5. Conclusions

This paper proposed a novel APA based on the adjustment of input-vector number to enhance the filter performance in aspects of both convergence rate and steady-state estimation error. In this framework, the input-vector number for the proposed APA was determined using the pseudo-fractional method. Because the pseudo-fractional method relaxed the constraint of the conventional APA that required the input-vector number to be integer, it was meaningful to decide the input-vector number more precisely to improve the filter performance. Specifically, the proposed method dynamically adjusted the input-vector number by using the proposed adaptation rule for the fractional input-vector number. According to the adaptation rule, the current projection order was set by comparing the AASE values reasonably. Simulation results with system identification and speech input scenarios demonstrated that the proposed APA had smaller steady-state estimation error as compared to the existing APA-type algorithms. Moreover, we plan to develop an improved APA to adjust both the projection order and step size based on the proposed method. Because the E-APA [16] was expanded by using the variable projection order and step-size concept simultaneously to enhance the filter performance in aspects of convergence rate and steady-state estimation error, it can be an effective way for our proposed APA to improve the filter performance. By extension, we will research the expansion for various kinds of adaptive filter through the variable projection-order and step-size strategies to improve the filter performance.