The work presented in this paper deals with object tracking problem which has vast application prospects. Tracking of surrounding moving objects is important for the development of a navigation system with proper planning and motion techniques. Because of its accurate estimation characteristic, we have used Kalman filter to estimate the motion parameters of a moving object and then fuzzy logic technique is used to improve the performance of Kalman filter. The proposed fuzzy logic controller is a multiple input system with nine rules to adapt noise covariance matrices based on the innovation sequence of a Kalman filter. We have compared the results between conventional Kalman filter and fuzzy aided Kalman filter. The software based simulation results verified that this unique fuzzy aided Kalman filter has a positive effectiveness for the object tracking problem over conventional Kalman filter because of its capability to recover filter divergence problem.
Object tracking is one of the most attractive research areas in engineering field. Estimation of the locations and motion parameters of a moving target in a predictable or partially predictable manner is the goal of object tracking. Such techniques help us to understand the orientation and structure of an object. Object tracking has vast application prospects in the field of navigation [1]. The navigation of a vehicle or a robot needs a path planning which ensures a collision free path from the starting location to the target location. Thus, detection of the surrounding moving objects or obstacles is necessary for a successful path planning. Tracking of moving object is also important for military guidance, traffic monitoring [2], safety detection and medical and biological applications.
Moving object tracking is an important, yet challenging task to accomplish. Many researchers employ various approaches in moving object tracking depending on the application domain. In [3], the author proposed an optical Kalman filter for object tracking. Prakash et al. [4] proposed dual tree complex wavelet transform algorithm for moving object tracking in video sequence. Some researchers have used probability method such as Kalman filter and particle filter for object tracking. Banarjee and Pallipuram [5] also used Kalman filter for multi person tracking. However using only single algorithm for object tracking can give unsatisfactory results because every algorithm has its own limitations. So, combined algorithms are proposed by researchers to minimize the drawbacks of individual algorithm. Chen et al. [6] combined particle filter with mean-shift for video motion tracking in order to improve the performance of particle filters. Extended Kalman filter was coupled with multiple models in [7] to track the target.
Kalman filter is widely used to track moving objects since it has the ability to adapt to nonstationary environments [8]. For the object to be tracked the assumption of motion is important in order to ensure accurate estimation by Kalman filter. Tuning of measurement noise (observation noise) and process noise (plant noise) plays a vital role in estimation of the system dynamics accurately. Fuzzy logic is one of the most popular optimization techniques which can be incorporated into Kalman filter to tune measurement noise and plant noise.
The Kalman filtering and the fuzzy logic have been found of respective superiority in different applications. Fuzzy theory has the ability to imitate human being’s subjective decision. With fuzzy theory, smart systems can be made to adapt to their environment. This is why researchers are trying to use fuzzy logics to adapt and overcome the shortcomings of Kalman filter. In [9], the authors proposed a zero phase adaptive fuzzy Kalman filter in order to suppress vibration of a robotic surgical system. Alqudaihi et al. [10] presented a hyper fuzzy extended Kalman filter for indoor security monitoring system. Li et al. [11] has worked on combining fuzzy adaptive strategy with strong tracking scent unscented Kalman filter to solve the problem of misalignment angles in inertial navigation system. Tseng et al. [12] has presented a sensor fusion method combining cubature Kalman filter (CKF) and fuzzy logic adaptive system (FLAS) for the integrated navigation systems. All of the above mentioned works have applied fuzzy logic aided Kalman filter in different application areas. In our work, we have used Madmani based fuzzy logic to adapt a linear Kalman filter which can solve an object tracking problem with improved performance.
Escamilla-Ambrosio and Mort [13] developed a SISO fuzzy inference system with three rules to tune Kalman filter noise parameters. Based on that, we have incorporated a multiple input fuzzy logic into Kalman filter algorithm with nine rules for 2D tracking of a moving object. Fuzzy aided Kalman filter has estimated the three components of position and the three components of velocity which make up a six-component of state vector fully describing the motion of the object. We have also observed the estimation of these states with both conventional Kalman filter and fuzzy aided Kalman filter. Finally, we have compared the results of our fuzzy aided Kalman filter with those obtained from a conventional Kalman filter and in all the cases, adaptive Kalman filter gives better result than conventional Kalman filter.
This paper is divided into following sections: Section 2 describes the basics of the Kalman filter algorithm and fuzzy logic system. Section 3 consists of modeling and validation of the system. Section 4 contains the simulation results and Section 5 outlines the conclusion.
A Kalman filter is an optimal recursive estimator which estimates the state of a dynamic system [14]. Based on the current state and current uncertainty of measurements, Kalman filter predicts the next state of a given system. Kalman filter used state space model for linear estimation. The state of a discrete-time controlled estimation process is governed by the following linear stochastic difference
where k = 1, 2 …, x_{k} is an (n × 1) system state vector, A_{k} is an (n × n) transition matrix which represent the relationship between current system state and last system state, u_{k} is an (l × 1) vector of the input forcing function, B_{k} is an (n × l) matrix representing the relationship between control process and current system states, w_{k} is an (n × 1) process noise vector.
The measurement vector is given by the following equation:
where z_{k} is an (m × 1) measurement vector, H_{k} is an (m × n) measurement matrix which maps system state to measurement, v_{k} is an (m × 1) measurement noise vector.
In order to apply Kalman filter solution, we assume that average value of both process noise vector and measurement noise vector is zero and no correlation exists between them. Based on these assumptions we can define process noise covariance matrix Q_{k} and measurement noise covariance matrix R_{k} as follows:
where E{·} is the statistical expectation, superscript T denotes transpose. After defining all the required matrices for Kalman filter, now we can move on to Kalman filter algorithm equations. Kalman filter algorithm has two stages known as:
Prediction stage characterized by time updates equations:
From
Second step of Kalman filter is known as correction stage given by the following equations:
From
Fuzzy set theory was developed by Professor L. A. Zadeh [15] in 1965. It resembles human reasoning and provides a formal methodology to represent, manipulate and implement a human’s heuristic knowledge to control a system. It is a knowledge based system which produces acceptable definite output in response to incomplete or inaccurate (fuzzy) input. It consists of linguistic IF-THEN rules which is generally expressed in the form: IF variable is property THEN action. The steps of fuzzy controller system design are basically accomplished as (1) determining the fuzzification method and selecting membership functions for both inputs and outputs. (2) Determining the fuzzy inference system and documenting the rule set via a set of IF-THEN expressions and (3) determining the defuzzification method. Fuzzy systems are suitable for the modeling of vague knowledge.
In this section we will formulate the system model. First we need to define the system states. The dynamics of any moving object can be described by the following equations:
where x(k) and v (k) denotes the position and velocity at time step k and a denotes the acceleration. Since our target is to track the movement of an object in a specified time interval T in a plane, we will assume that the motion of the object is defined by 3 variables: horizontal translation or translation on the x-axis, y-axis, and z-axis. We will estimate the 3D coordinates of the position vector and velocity vector. We will also track the trajectory of the moving object in two-dimensional spaces. So in our system the state X (k) has the following variables of interest containing the positional data with its first derivatives (velocity):
where (x, y, z) is the position of the object at every moment of time within the time interval T. Based on
So we can write the state space model of our system as following:
where
And
where
Now if we introduce a white noise to the model we can write:
where w is known as the process noise. Here the covariance matrix of process noise is called Q which is given by the following equation:
In our system, w is uncorrelated zero-mean Gaussian white noise sequence with covariance matrix Q, where
The kinematics of the object is described by Gaussian white random sequences with variances of 0.1 cm^{2} in position (in x, y, z direction) and 0.01 cm^{2}s^{−2} in velocity (in x, y, z direction) as indicated by process noise covariance matrix. The off-diagonal elements of Q are zeros due to our assumption that the motion on the x-axis, y-axis and z-axis are uncorrelated.
v is uncorrelated zero-mean Gaussian white noise sequence with covariance matrix R, where
If the theoretical and actual behaviors of a Kalman filter do not agree with each other, divergence problems will occur. In order to estimate system states accurately, Kalman filter needs to assume all the system dynamics and noise processes. The validity of the estimated states largely depends on process noise covariance matrix Q and measurement noise covariance matrix R, respectively. Wrong assumption of priori information is also one of the main reasons of practical divergence of the filter. But to design a conventional Kalman filter in a dynamic environment can be proved as a setback for system states estimation. Because of all these reasons an adaptive estimation formulation of the Kalman filter is necessary to prevent filter divergence and ensure better performance.
We have implemented innovation based adaptive estimation technique which corrects the Kalman filter by adjusting the process noise covariance matrix Q and measurement noise covariance matrix R based on the available information obtained from the filter innovation sequence. The predicted residual vector or innovation sequence of a Kalman filter is expressed as
The theoretical covariance of innovation sequence is found from the following equation
It can be said that the diversion of the innovation sequence from a certain theoretical covariance range will ultimately result in a reduction of the filter performance. The good estimation of Q and R can ensure the consistency of innovation sequence with their theoretical covariance which is the basic idea of our adaption technique. So, if disperencies are noticed between the actual covariance and its theoretical value, then we will have to adjust R and Q in order to correct this mismatch. The mismatch between the actual innovation process and its theoretical value can be monitored by a variable named as degree of matching (DoM) which is given by the following equation.
S_{k} is the theoretical covariance of innovation sequence and S_{rk} is the actual covariance of innovation sequence which can be given by the following equation
where p_{0} = k − N + 1 is the first sample inside the estimation window, N. Here, N is chosen to give some statistical smoothing.
Our implemented fuzzy system to minimize this mismatch is a two-inputs-one-output Madmani type fuzzy logic controller which uses nine rules. First we have attempted (1) adaptive estimation of the measurement noise covariance matrix R with Q fixed and then we have implemented (2) adaptive estimation of process noise covariance matrix Q with fixed R.
Since measurement noise covariance matrix R represents the accuracy of the measurement instrument, an increase in R means we trust the measurement less. As a result, the filter converges slowly to the actual position. If R is decreased, it will converge to the actual position faster with frequent oscillations due to our trust on noisy measurements. We can say from
The proposed fuzzy inference system comprises of two inputs: the theoretical covariance of innovation sequence S_{k} and the actual covariance of innovation sequence S_{rk}. It has one output which is AdjR. The proposed adaptive formulation adjust the measurement error covariance R based on the following equation:
where AdjR is the FIS output which is added or subtracted from R.
Our fuzzy logic based controller uses 9 rules, such as:
If S_{rk} is Negative Small and S_{k} is Large then Decrease R.
The entire rule base of the proposed fuzzy inference system is shown in the following Table 1.
The membership function of input and output variables are Gaussian and triangular, respectively. They are shown in Figures 1 and 2.
The value of Q and R was fixed as described in Section 3, however the value of R was continuously adjusted using Fuzzy rules in order to get the accurate estimation from Kalman filter by correcting the mismatch between the actual innovation process and its theoretical value once the first value of S_{rk} was available.
Assuming that the noise covariance matrix R is completely known, an algorithm to estimate matrix Q can be derived based on the following equation:
We can say from the above equation, if Q increase then S will increase and vice versa. Since the relationship between Q and S is similar to relation between R and S, the same logic table can be used to adjust Q. The proposed adaptive formulation adjusts the measurement error covariance Q based on the following equation:
R was fixed as described in Section 3. The starting value of Q was selected to be Q_{0} = 5Q, however the value of Q was continuously adjusted using fuzzy rules in order to get the accurate estimation from Kalman filter by correcting the mismatch between the actual innovation process and its theoretical value once the first value of S_{rk} was available.
We have used MATLAB to simulate the adaptive Kalman filter to solve our object tracking problem and the results are shown in Section 5.
In this section, we provide simulation results of our study. We have used both Kalman filter and fuzzy aided Kalman filter to estimate the states of our problem and simulate our results. The trajectory of the moving object obtained from simulation studies are shown in Figures 3
In Figures 6
The error analysis and comparison of errors between Kalman filter and fuzzy aided Kalman filter (with adjusted R) are shown in Tables 2
The six states estimation results comparison for Kalman filter and fuzzy aided Kalman filter with adjusted Q are shown from Figures 12
In this paper, an object tracking algorithm which combines several conventional algorithms such as fuzzy logic and Kalman filter is developed. The purpose of this combination is to use the fuzzy logic controller in order to cover the drawbacks of Kalman filter. In the proposed method Kalman filter is used as the main tracer and fuzzy logic is used to recover the divergence problem of Kalman filter. In this way, we were able to increase the accuracy of the system significantly. From the simulation results it is proven that the proposed method is more suitable for real time implementation compare to the conventional Kalman filter algorithm. The use of multiple inputs instead of single input fuzzy logic controller with only nine rules will be helpful to address more sophisticated systems in future. The proposed algorithm can be used to track moving objects from a video sequence by representing the movie in a 2D array of matrix form and using state space model of moving objects on 2D camera images.
No potential conflict of interest relevant to this article was reported.
No potential conflict of interest relevant to this article was reported.
Membership functions: (a) input membership function
Output membership function,
Trajectory of the moving object (actual state and Kalman filter estimated state).
Trajectory of the moving object (actual state and fuzzy aided Kalman filter estimated state when
Trajectory of the moving object (actual state and fuzzy aided Kalman filter estimated state when
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Comparison of Kalman filter (KF) and fuzzy aided Kalman filter (FAKF) (with adjusted
Fuzzy rule base for adjusting
ZE | NS | L | |
---|---|---|---|
ZE | M | I | D |
NS | D | M | D |
L | I | I | M |
ZE, zero; NS, negative small; L, large; M, maintain; I, increase; D, decrease
Error analysis in
Tme frame (s) | Actual | Estimated | Error (cm) | Actual | Estimated | Error (cm) | Actual | Estimated | Error (cm) |
---|---|---|---|---|---|---|---|---|---|
5–10 | 116.50 | 99.74 | 16.76 | 1055.7 | 1027.4 | 28.30 | 118.65 | 107.59 | 11.06 |
10–15 | 575.29 | 424.66 | 150.63 | 3122.8 | 3054.3 | 68.50 | 591.05 | 533.75 | 57.30 |
15–20 | 1783.0 | 1580.5 | 202.5 | 4241.6 | 4086.2 | 155.4 | 1837.0 | 1645.5 | 191.5 |
20–25 | 3718.9 | 3457.4 | 261.5 | 4430.2 | 4072.2 | 358.0 | 3822.4 | 3649.4 | 273.0 |
25–30 | 4760.6 | 4435.1 | 334.5 | 4222.6 | 3818.2 | 404.4 | 4890.2 | 4543.5 | 346.7 |
30–35 | 5744.9 | 5389.6 | 355.3 | 3908.2 | 3459.0 | 449.2 | 5876.6 | 5492.9 | 383.7 |
35–40 | 8718.9 | 8283.1 | 435.8 | 2533.3 | 1953.8 | 579.5 | 8876.1 | 8386.3 | 489.8 |
Error analysis in
Tme frame (s) | Actual | Estimated | Error (cm) | Actual | Estimated | Error (cm) | Actual | Estimated | Error (cm) |
---|---|---|---|---|---|---|---|---|---|
5–10 | 116.50 | 111.91 | 4.59 | 1055.7 | 1065.3 | −9.60 | 118.65 | 115.39 | 3.26 |
10–15 | 575.29 | 538.63 | 36.66 | 3122.8 | 3128.9 | −6.10 | 591.05 | 555.45 | 35.6 |
15–20 | 1783.0 | 1748.7 | 34.30 | 4241.6 | 4206.2 | 35.40 | 1837.0 | 1815.7 | 21.3 |
20–25 | 3718.9 | 3673.2 | 45.70 | 4430.2 | 4406.6 | 23.60 | 3822.4 | 3812.0 | 10.4 |
25–30 | 4760.6 | 4759.8 | 9.80 | 4222.6 | 4220.8 | 1.80 | 4890.2 | 4905.7 | −15.5 |
30–35 | 5744.9 | 5754.1 | −9.2 | 3908.2 | 3944.4 | −36.2 | 5876.6 | 5880.2 | −3.60 |
35–40 | 8718.9 | 8716.4 | 2.5 | 2533.3 | 2520.1 | 13.20 | 8876.1 | 8836.3 | 39.80 |
Error analysis in
Tme frame (s) | Actual | Estimated | Error (cm/s) | Actual | Estimated | Error (cm/s) | Actial | Estimated | Error (cm/s) |
---|---|---|---|---|---|---|---|---|---|
5–10 | 5.61 | 17.83 | −12.2 | 358.23 | 356.30 | 1.93 | 18.74 | 12.32 | 6.42 |
10–15 | 104.06 | 123.14 | −19.0 | 237.22 | 240.94 | −3.72 | 115.87 | 102.95 | 12.92 |
15–20 | 208.51 | 229.31 | −20.8 | 143.45 | 152.11 | −8.66 | 167.32 | 156.34 | 10.98 |
20–25 | 225.20 | 238.33 | −13.1 | 87.29 | 94.02 | −6.73 | 267.14 | 241.23 | 25.91 |
25–30 | 362.12 | 386.65 | −24.5 | −55.38 | −41.72 | −13.66 | 365.21 | 348.10 | 17.11 |
30–35 | 398.39 | 419.42 | −21.0 | −176.7 | −154.87 | −21.83 | 415.13 | 402.67 | 12.46 |
35–40 | 501.53 | 512.31 | −10.7 | −277.2 | −256.17 | −21.03 | 512.23 | 498.14 | 14.09 |
Error analysis in
Tme frame (s) | Actual | Estimated | Error (cm/s) | Actual | Estimated | Error (cm/s) | Actial | Estimated | Error (cm/s) |
---|---|---|---|---|---|---|---|---|---|
5–10 | 5.61 | 6.27 | −0.66 | 358.23 | 357.28 | 0.95 | 18.74 | 15.87 | 2.87 |
10–15 | 104.06 | 101.03 | 3.03 | 237.22 | 234.65 | 2.57 | 115.87 | 112.84 | 3.03 |
15–20 | 208.51 | 209.45 | −0.94 | 143.45 | 146.23 | −2.78 | 167.32 | 164.00 | 3.32 |
20–25 | 225.20 | 314.40 | 0.81 | 87.29 | 89.07 | −1.78 | 267.14 | 263.34 | 3.80 |
25–30 | 362.12 | 361.74 | 0.38 | −55.38 | −54.98 | −0.40 | 365.21 | 364.76 | 0.45 |
30–35 | 398.39 | 396.78 | 1.61 | −176.7 | −176.55 | −0.15 | 415.13 | 413.90 | 1.23 |
35–40 | 501.53 | 500.72 | 0.81 | −277.2 | −281.71 | 4.51 | 512.23 | 509.54 | 2.69 |
E-mail: hamidrawan92@gmail.com
E-mail: azzama.eee@aust.edu
E-mail: ehtesan@aust.edu