SugenoYasukawa fuzzy modelling: survey and improvements

Domonkos Tikk and Gyrgy Bir o o Dept. of Telecommunication & Telematics Budapest Univ. of Technology and Economics Budapest, Hungary e-mail: {tikk}@ttt.bme.hu

2nd International Symposium of Hungarian Researchers

Overview
1 2 3 4 Motivations Description of the SY method Improvements on implementation details Conclusions, further works and results

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Motivation
Aiming at creating hierarchical fuzzy systems based on input-output sample data set Connecting sparse rule based techniques and hierarchial reasoning methods SY method is a good starting point, because it does not create a full rule base, but usually a sparse one, hence the number of rules are normally small.

2
2.1

The SY method
Overall description of tasks
The goal of the SY fuzzy modelling method is to create a transparent, and linguistic interpretable fuzzy rule based model from input-output sample data. Two main steps: identication and the build-up of the qualitative model. Identication is subdivided into two tasks: the structure identication and parameter identication.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Structure identication tasks are of two types: Type I includes: nding the input candidates of the system and 2. nding its actual variables which aect the output. Type II includes: the determination of the number of rules and the partition of the (usually) multidimensional input space.

Classication of identication Structure identication I a: selection of the possible input candidates b: determination the true inputs (feature selection) Structure identication II a: determination of the number of the rules b: partition of the input space Parameter identication

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

2.2

Feature selection
Goal: choose a set of eective variables among a nite set of original variables.

Evaluation: Candidate sets are evaluated by means of a criterion function Tool: SY method uses the regularity criterion (RC). It checks at most n(n + 1)/2 possible input candidates sets. The RC is a heuristic method which selects a set of inputs among the possible candidates. In the rst step, the sample data set is divided into two groups, A and B. The criterion function is
kA kB

kA and kB are the numbers of data in groups A and B; yiA and yiB are the outputs of groups A and B;

RC =

(yiA yiAB )2/kA +

i=1

i=1

(yiB yiBA)2/kB 2

yiAB (yiBA) is the model output for the group A (B) input estimated by the model identied using group B (A) data.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

2.3

Structure identication

Reverse order: starting from the rule consequents then nding a partition concerning the antecedents. The partition of the input space is not full to avoid computational explosion. Fuzzy c-means (FCM) clustering of the output data set. From the obtained results the partition of the output space can be approximated. Then the output clusters are projected into the input space, and the parameters of the input clusters are determined (in each dimension). The number of the output clusters are determined by the following evaluation function
N C

S(C) =
k=1 i=1

(ik )m

xk vi

vi x

while S(c) decreases the performance increases.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Figure 1: Full partition of the space in 2 dimension

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Figure 2: Projection of the output clusters to the input space

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Figure 3: Projection of an input cluster to the input dimensions

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Figure 4: When more than one input cluster belongs to one output cluster

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

2.4

Parameter identication
N

Parameter identication step is accomplished by evaluating the following performance index. PI =

i=1

(y i y i)2/N

(1)

where y i is the model output for the ith sample datum. In the case of fuzzy model, the parameters are those of the membership functions. Having trapezoidal membership functions it means 4 parameter for each antecedent p 1 p2 p3 p4. In the parameter identication step they adjusted these four values in an iterative algorithm. 5% of the width of the actual input space was applied as adjusting value.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o 2001

2nd International Symposium of Hungarian Researchers

Start Set number of cluster c = 2 c=c+1

?
Make fuzzy clustering

?
No S(c) minimal

Yes

Find the parameters B

?
Divide data into two gropus

?
Assume single variables among input candidates

?
Find the parameters A with respect to each data group

?
Add a new input variable No Calculate RC for each input variable

?
RC increases?

? Yes
Combine two groups of data to reform the parameters A Initialize position gradient type model

?
Make parameters identication

?
End

2nd International Symposium of Hungarian Researchers

3
3.1

Improvements
Trapeze approximation
1. First, the convex hull of the original data set is determined. 2. Second, the convex hull is approximated by a trapezoidal membership function. We proposed a simple and fast trapeze approximation algorithm in three dierent versions. The three versions dier in the determined support length, or in other words, in the angle of the slopes of the trapeze. The dierence is signicant when the distribution of the data with high membership grades is large near the minimum/maximum of the cluster. In such cases the rst version results in close-to-oblong shape trapezes with steep slopes, the third in trapezes with long and smooth slopes, while the second version generates an average solution of the former two. The trapeze approximation of the clustered raw data is proceeded in two steps.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

(a)

(b)

(c)

Figure 5: The two steps of trapezoidal membership function construction. (a) c-means clustered raw data; (b) the convex hull of the input cluster; (c) approximated trapezoidal fuzzy set

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Trapeze approximation: 1. Determine min and max the minimum/maximum of the membership degree of the data point in the given cluster, xmax the rst point where the maximum max is attained, further dmin and dmax the minimum/maximum of all the data values in the given cluster domain. 2. Set the boundaries of investigated interval to r1 + max/r r r1 + min/r M = max r where r > 1 is a given parameter, usually between 2 and 4. m = min 3. Determine the parameters of the left slope, x1 and x2, as: (a) Let us initialize xj as the last data point which has smaller membership degree than m and xi be the next point of the convex hull: (xj ) < m; (xj+1 ) = (xi) > m. (2)

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

(If there is no such point x in the convex hull which satises (x) < m then x j and xi is the rst two leftmost point of the convex hull). Further the parameters of the left x1 = dmin and x2 = dmin . (b) Let x and x be the location of the intersection made by the support and the core, 1 2 respectively, with the line passing through the points (x j , (xj )) and (xi, (xi)) (see Figure 6). (c) If x > x1 then x1 := x. 1 1 (d) If x < x2 or i = j + 1 then x2 := x. 2 2 (e) If xi+1 xmax then let i := i + 1 and go to step (3b), otherwise continue.
=1

....................................... .......................................
3 6 I ]

M = 2/3 range m = 1/3 range -

x 1

xj xi

x 2

Figure 6: Determination of the location x and x . Here r = 3. 1 2

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

4. Determine the parameters of the left slope, x3 and x4, analogously as in the previous step. 5. Order the parameters according to x1 x2 x3 x4.

3.2

Critiques on the RC method

very sensible for data grouping at each step it requires the model construction cycle. sensible for the rule parameters determined by the projection of the input data. We decided to look for another feature selection method. However, in the literature we found only feature ranking methods for classication purposes, so we modied one of them, the interclass separability based algorithm and adopted it for fuzzy modelling problems.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

3.3

Determining the number of the rules

This is an important detail of the original algorithm in our view, which was not given the required attention in the original paper. This question has signicant eect on the total number of rules and, thus, on the computational complexity of the nal model. The method plays an important role especially, when the modelled function is not strictly monotone, unlike the examples of the original paper, but the same output is assigned to several region of the input space.
9 8
1.5 2 2

7 6 5 4 3 2 1 5 4 3 3 2 x 1 1
2

output: (1+x1 +x2

5 4 2 x
1

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Algorithm for determining the antecedents Inputs: training data set, membership degree vector of the fuzzy c-means clustered training data set, output: the actual rules. 1. Select a cluster i, where i = 1, . . . , C, and divide the training data set into two groups: group A consists of datum with membership degree not smaller than 0.5 in cluster i, while group B includes the remaining data set: A = {xj |i(xj ) 0.5, j = 1, . . . , N } B = {xj |i(xj ) < 0.5, j = 1, . . . , N } 2. Create a rule base R of 2 |A| dierent rules, by forming separate rules from each of the group A data end the rst |A| group B data. If |A| > |B| then we form N rules. At this point the rules are crisp in the sense that the antecedents as well as the consequent is a crisp value. 3. Evaluate the performance of this rule base by checking this model on the group B data set. We determine the rule base performance by counting the data with incorrectly high
D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o 2001

2nd International Symposium of Hungarian Researchers

membership grade as: pR = |B |/|B| where B = {xj B|WR (xj ) t1, j = 1, . . . , N } Here function W (W : Rn [0, 1]) determines the ring weight of the datum xj by the rule base R, and the t1 is a given threshold, usually 0.5. 4. Temporarily merge the ith and the jth rules by using the and operation for the corresponding antecedents and the consequents. Denote the obtained rule base by R . Calculate pR . If pR > p R + t 2 then the performance of the new rule base is signicantly worse, therefore undo the fusion of the two rules; otherwise merge them permanently: R := R . Here t2 is another threshold parameter, usually 0.5. 5. If all the rule pairs is checked then stop, otherwise check the next pair and go to step 4.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001

2nd International Symposium of Hungarian Researchers

Conclusions
We cleared some vague unexplicit details of the Sugenos and Yasukawas qualitative fuzzy modelling method. We proposed algorithms for trapeze approximation, for the determination of the number of rules (belonging to one output cluster) and parameter identication. We investigated the reliability and stability of the RC method, which is applied to determine the true inputs among input candidate variables, and we found that result were not satisfactory.

D. Tikk and Gy. Bir: SugenoYasukawa fuzzy modelling: survey and improvements o

2001