The Effect of Decomposition Method as Data Preprocessing on Neural Networks Model for Forecasting Trend and Seasonal Time

Series
Suhartono
Statistics Department, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia Email: suhartono@statistika.its.ac.id; har arema@yahoo.com

Subanar
!athematics Department, "ni#ersitas $ad%ah !ada, &ogyakarta, Indonesia Email: subanar@yahoo.com

ABSTRACT
Recently, one of the central topics for the neural networks (NN) community is the issue of data preprocessing on the use of NN. In this paper, we will investigate this topic particularly on the effect of Decomposition method as data processing and the use of NN for modeling effectively time series with both trend and seasonal patterns. imited empirical studies on seasonal time series forecasting with neural networks show that some find neural networks are able to model seasonality directly and prior deseasonali!ation is not necessary, and others conclude "ust the opposite. In this research, we study particularly on the effectiveness of data preprocessing, including detrending and deseasonali!ation by applying Decomposition method on NN modeling and forecasting performance. #e use two kinds of data, simulation and real data. $imulation data are e%amined on multiplicative of trend and seasonality patterns. &he results are compared to those obtained from the classical time series model. 'ur result shows that a combination of detrending and deseasonali!ation by applying Decomposition method is the effective data preprocessing on the use of NN for forecasting trend and seasonal time series. Keywords( decomposition, data preprocessing, neural networks, trend, seasonality, time series, forecasting.

1. NT!"D#$T "N !any business and economic time series are non'stationary time series that contain trend and seasonal #ariations. The trend is the long'term component that represents the gro(th or decline in the time series o#er an e)tended period o* time. Seasonality is a periodic and recurrent pattern caused by *actors such as (eather, holidays, or repeating promotions. +ccurate *orecasting o* trend and seasonal time series is #ery important *or e**ecti#e decisions in retail, marketing, production, in#entory control, personnel, and many other business sectors ,!akridakis and -heel(right, ./012. Thus, ho( to model and *orecast trend and seasonal time

S"3+4T5N5 +ND S"6+N+4

series has long been a ma%or research topic that has signi*icant practical implications. There are some *orecasting techni7ues that usually used to *orecast data time series (ith trend and seasonality, including additi#e and multiplicati#e methods. Those methods are -inter8s e)ponential smoothing, Decomposition, Time series regression, and +4I!+ models ,see e.g. 6o(erman and 589onnel ,.//:2 or 3anke and 4eitsch ,.//;22. 4ecently, Neural Net(orks ,NN2 models are also used *or time series *orecasting ,see e.g. <ara(ay and 9hat*ield ,.//02, 3ill et al. ,.//=2, also >aashoek and ?an Di%k, ,@AA.22. Suhartono et al. ,@AA;2 did comparati#e study o* these methods by using airline data and concluded that there (as no best model satis*ies simultaneously in both training and testing data. They also recommended the possibility *or doing *urther research by combining some methods. The aim o* this paper is to de#elop ne( hybrid model by combining decomposition method as data preprocessing and NN model *or *orecasting trend and seasonal time series. The results are compared to +4I!+ models. %. M"DE& N' T!END (ND SE(S"N(& T ME SE! ES !odeling trend and seasonal time series has been one o* the main research endea#ors *or decades. In the early ./@As, the decomposition model along (ith seasonal ad%ustment (as the ma%or research *ocus due to Bersons ,././, ./@:2 (ork on decomposing a seasonal time series. 3olt ,./;12 and -inters ,./=A2 de#eloped method *or *orecasting trend and seasonal time series based on the (eighted e)ponential smoothing. +mong them, the (ork by 6o) and Cenkins ,./1=2 on the seasonal +4I!+ model has had a ma%or impact on the practical applications to seasonal time series modeling. This model has per*ormed (ell in many real (orld applications and is still one o* the most (idely used seasonal *orecasting methods. !ore recently, NN ha#e been (idely used as a po(er*ul alternati#e to traditional time series modeling ,see e.g. 3ansen and Nelson ,@AA:2, Nelson et al. ,.///2, also Dhang et al. ,.//022. -hile their ability to model comple) *unctional patterns in the data has been tested, their capability *or modeling seasonal time series is not systematically in#estigated. In this section, (e (ill gi#e a brie* re#ie( o* these *orecasting models, particularly seasonal +4I!+, decomposition method and NN model. %.1. Seasonal (! M( Model The seasonal +4I!+ model belongs to a *amily o* *le)ible linear time series models that can be used to model many di**erent types o* seasonal as (ell as nonseasonal time series. The seasonal +4I!+ model can be e)pressed as ,see e.g. 6o) et al. ,.//E2, 9ryer ,./0=2, and -ei ,.//A22:
2

The E**ect o* Decomposition !ethod as Data Breprocessing on Neural Net(orks F

$ p , , 2+ , , $ 2,. , 2 d ,. , $ 2 D yt =* , , 2 ) , , 2t ,

,.2

(here $ is the seasonal length, , is the back shi*t operator and t is a se7uence o* (hite noises (ith Gero mean and constant #ariance. 6o) and Cenkins ,./1=2 proposed a set o* e**ecti#e model building strategies *or seasonal +4I!+ based on the autocorrelation structures in a time series. %.%. Decomposition Method The multiplicati#e decomposition model has been *ound to be use*ul (hen modeling time series that display increasing or decreasing seasonal #ariation ,6o(erman and 589onnel, .//:; chapter 12. The key assumption inherent in this model is that seasonality can be separated *rom other components o* the series. The multiplicati#e decomposition model is
y t = &t $ t - t I t

,@2

(here
yt &t $t -t It

H H H H H

the obser#ed #alue o* the time series in time period the trend component in time period t the seasonal component in time period t the cyclical component in time period t the irregular component in time period t .

%.). Neural Networks Model Neural net(orks ,NN2 are a class o* *le)ible nonlinear models that can disco#er patterns adapti#ely *rom the data. Theoretically, it has been sho(n that gi#en an appropriate number o* nonlinear processing units, NN can learn *rom e)perience and estimate any comple) *unctional relationship (ith high accuracy. Empirically, numerous success*ul applications ha#e established their role *or pattern recognition and time series *orecasting. <eed*or(ard Neural Net(orks ,<<NN2 is the most popular NN models *or time series *orecasting applications. <igure . sho(s a typical three'layer <<NN used *or *orecasting purposes. The input nodes are the pre#ious lagged obser#ations, (hile the output pro#ides the *orecast *or the *uture #alues. 3idden nodes (ith appropriate nonlinear trans*er *unctions are used to process the in*ormation recei#ed by the input nodes. The model o* <<NN in *igure . can be (ritten as
* p yt = A + " f i" y t i + o" + t , " =. i =.

,:2

S"3+4T5N5 +ND S"6+N+4

(here p is the number o* input nodes, * is the number o* hidden nodes, f is a sigmoid trans*er *unction such as the logistic:
f , %2 = . . + e %

,E2

Note that e7uation ,:2 indicates a linear trans*er *unction is employed in the output node.

J " , " = A,., , *I is a #ector o* (eights *rom the hidden to output nodes and J i" , i = A,., , p; " =.,@, , *I are (eights *rom the input to hidden nodes.

i"

"
Output Layer (Dependent Var.)

Input Layer (Lag Dependent Var.) Hidden Layer (q unit neurons)

Kt y

Figure 1. Architecture of neural network model with single hidden layer <unctionally, the <<NN e)pressed in e7uation ,:2 is e7ui#alent to a nonlinear +4 model. This simple structure o* the net(ork model has been sho(n to be capable o* appro)imating arbitrary *unction ,see e.g. 9ybenko, ./0/; 3ornik et al., ./0/a, ./0/b; and -hite, .//A2. 3o(e#er, *e( practical guidelines e)ist *or building a <<NN *or a time series, particularly the speci*ication o* <<NN architecture in terms o* the number o* input and hidden nodes is not an easy task.

). !ESE(!$* MET*"D"&"'+

The E**ect o* Decomposition !ethod as Data Breprocessing on Neural Net(orks F

The purpose o* this research is to pro#ide empirical e#idence on the comparati#e study o* many data preprocessing method in NN model *or *orecasting trend and seasonal time series. The ma%or research 7uestions (e in#estigate is: Does data preprocessing has a great impact on the accuracy o* NN model *or *orecasting trend and seasonal time seriesL -hich data preprocessing is the most e**ecti#e on NN model *orr *orecasting model *or trend and seasonal time seriesL -e conduct empirical study (ith simulation and real data, the international airline passenger data, to address these 7uestions. This real data has been analyGed by many researchers, see *or e)ample Nam and Schae*er ,.//;2, 3ill et al. ,.//=2, <ara(ay and 9hat*ield ,.//02, +tok and Suhartono ,@AAA2, Suhartono et al. ,@AA;a, @AA;b2. This data also has become one o* t(o data to be competed in Neural Net(ork <orecasting 9ompetition on Cune @AA; ,see (((.neural' *orecasting.com2. ).1. Data The simulation and real data contain .EE month obser#ations. The *irst .@A data obser#ations are used *or model selection and parameter estimation ,training data in term o* NN model2 and the last @E points are reser#ed as the test *or *orecasting e#aluation and comparison ,testing data2. <igure @ plots representati#e time series o* these data. It is clear that the series has an up(ard trend together (ith seasonal #ariations. ).%. !esearch Design Three types o* data preprocessing based on the decomposition method are applied and compare to the airline data. Those are detrend, deseasonal, and combination detrend'deseasonal. +ll o* these data preprocessing are implemented by using !INIT+6 so*t(are. To determine the best hybrid model, that is combination data preprocessing based on the decomposition method and NN model, an e)periment is conducted (ith the basic cross #alidation method. The a#ailable training data is used to estimate the (eights *or any speci*ic model architecture. The testing set is the used to select the best model among all models considered. In this study, the number o* hidden nodes #aries *rom . to .A (ith an increment o* .. The lags o* ., .@ and .: are included due to the results o* <ara(ay and 9hat*ield ,.//02, +tok and Suhartono ,@AAA2, and Suhartono et al. ,@AA;a2. The <<NN model used in this empirical study is the standard <<NN (ith single'hidden'layer sho(n in <igure .. -e use S'Blus to conduct <<NN model building and e#aluation. The initial #alue is set to random (ith ;A replications in each model to increase the chance o* getting the global minimum. -e also use the

S"3+4T5N5 +ND S"6+N+4

standard data preprocessing in NN *or the airline data by trans*orm detrend, deseasonal, and combination detrend'deseasonal data to N,A,.2 scale. The per*ormance o* in'sample *it ,training data2 and out'sample *orecast ,testing data2 is %udged by the commonly used error measures, the mean s7uared error ,!SE2 and ratio !SE to +4I!+ model.
Simulation data

Training data

Testing data

Airline passenger data

Training data

Testing data

Figure %. Time series plot of simulation and real data ,. EMP ! $(& !ES#&TS Table . summariGes the result o* the impact o* some data preprocessing on NN *orecasting and report per*ormance measures across training and testing samples *or the simulation data. Numbers greater than one on column ratio indicate poorer *orecast per*ormance than comparable +4I!+ model, and #ice #ersa *or numbers less than one.

The E**ect o* Decomposition !ethod as Data Breprocessing on Neural Net(orks F

Table 1. The result of the comparison between preprocessing data for FFNN and AR !A models" both in training and testing data" for the simulation data.
Model and Preprocessing IN-SAMPLE !"AININ# $A!A% MSE ARIMA model 0.0234 !2 "atio to A"IMA " O&!-SAMPLE !ES!IN# $A!A% MSE 0.020"""0 "atio to A"IMA "

##$$ model ("). %riginal Data a. Model 3&"&" ('') (. Model 3&"0&" (') (2). Detrend a. Model 3&2&" ('') (. Model 3&"0&" (') (3). Deseasonal . Model 3&3&"
('') (')

0.0"!3"23 0.00)*+03 0.0"!00+2 0.00 *!"3 0.))! 32! 0.00)"0 ) 0.003 444

0.!3+ 0.2)) 0.!2) 0.2*! 23.! 2 0.2"+ 0."))

0.02432+* 0.404"0!+ 0.02)24"" 0.0!22*)3 2.*)"!+) 0.00*4+4 4.30+++

".2"0 20.0*) ".2)) 3.)*) "4 .!+2 0.4!2 2"4.2

(4). Detrend&Deseasonal a. Model 3&)&" ('') (. Model 3&"0&" (')

'% ''%

, t-e (est model in training data (in&sample .ore/ast) , t-e (est model in testing data (out&sample .ore/ast)

The results o* the impact o* some data preprocessing on NN *orecasting and report per*ormance measures across training and testing samples *or the airline data are summariGed in table @. Se#eral obser#ations can be made *rom table . and @. <irst, detrend as data processing does yield poorer result than the original data or +4I!+. It can be clearly seen *rom table . and @ that the ratio !SE at testing samples *or NN are greater than .. Second, deseasonal as data processing gi#es the (orst result than other data preprocessing and also compared to +4I!+. -e can obser#e that the best model in testing samples by using deseasonal as data preprocessing yield the greatest ratio !SE compared ti the results o* the original data or the ratio o* detrend as data preprocessing. Third, the combination detrend'deseasonal as data preprocessing yields the best result *or *orecasting the airline data. It can be sho(n by the least ratio o* !SE at testing data. In general, (e can clearly see on the ratio o* testing samples comparison that combination detrend'deseasonal as data preprocessing and trans*ormation N,A,.2 on <<NN (ith ; unit nodes ,*or simulation data2 and E unit nodes ,*or the airline data2 in hidden layer yield the best !SE. The reduction o* !SE is highly signi*icant i* compare to the result o* <<NN (ithout detrend'deseasonal as data preprocessing, those are ;@.0M *or simulation data and =..0M *or the airline data.

S"3+4T5N5 +ND S"6+N+4

Table %. The result of the comparison between preprocessing data for FFNN and AR !A models" both in training and testing data" for the airline passenger data#
Model and Preprocessing IN-SAMPLE !"AININ# $A!A% MSE ARIMA model ++.+ "+ "atio to A"IMA " O&!-SAMPLE !ES!IN# $A!A% MSE ")2!.03 "atio to A"IMA "

##$$ model and data trans.orm to $(00") ("). %riginal Data a. Model 3&"&" ('') (. Model 3&"0&" (') (2). Detrend a. Model 3&4&" ('') (. Model 3&"0&" (') (3). Deseasonal a. Model 3& &" ('') (. Model 3&"0&" (') (4). Detrend&Deseasonal a. Model 3&4&" ('') (. Model 3&"0&" (')
'% ''%

*2.+!2* 2 .3230 !".0023 20.20)0 2).2444 "2.*04! 3).4 0+ "".3+42

".04) 0.2* 0.!** 0.22! 0.2+4 0."4) 0.3** 0."2+

"2"*.+" )2**.0 " !2.2! ) 30.3)

0.!** 3.4!0 ".0*) 3. +!

42"+."+ 2))*3*.30 )+2.*3 ")32."!

2.! 2 " !. 0* 0.3+2 ".003

, t-e (est model in training data (in&sample .ore/ast) , t-e (est model in testing data (out&sample .ore/ast)

-. $"N$&#S "NS 6ased on the results (e can conclude that the combination detrend' deseasonal ,based on the decomposition method2 as data preprocessing in <<NN yields a great impact on the increasing accuracy o* *orecasting trend and seasonal time series. 5ur result also sho(s that the best model in training data tends to yield o#er*itting on testing. This condition gi#e a chance to do *urther research by implementing some NN model selection methods in order *or the model selection process becomes e**icient.

!EFE!EN$ES

The E**ect o* Decomposition !ethod as Data Breprocessing on Neural Net(orks F

+tok, 4.!. and Suhartono, @AAA. -omparison between Neural Networks, .RI/. ,o%01enkins and 2%ponential $moothing /ethods for &ime $eries 3orecasting, 4esearch 4eport, Nemlit: Institut Teknologi Sepuluh Nopember. 6o), $.E.B. and $.!. Cenkins, ./1=. &ime $eries .nalysis( 3orecasting and -ontrol, San <ransisco: 3olden'Day, 4e#ised edn. 6o), $.E.B., $.!. Cenkins and $.9. 4einsel, .//E. &ime $eries .nalysis, 3orecasting and -ontrol, :rd edition, Engle(ood 9li**s: Brentice 3all. 6o(erman, 6.N. and 4.T. 589onnel, .//:. 3orecasting and &ime $eries( .n .pplied .pproach, 4rd ed, 6elmont, 9ali*ornia: Du)bury Bress. 9ryer, C.D., ./0=. &ime $eries .nalysis, 6oston: B-S'>ENT Bublishing 9o. 9ybenko, $., ./0/. O+ppro)imation by superpositions o* a sigmoidal *unctionP, /athematics of -ontrol, $ignals and $ystems, %, :AEQ:.E. <ara(ay, C. and 9. 9hat*ield, .//0. OTime series *orecasting (ith neural net(ork: a comparati#e study using the airline dataP, .pplied $tatistics, ,., @:.Q@;A. <ildes, 4. and S. !akridakis, .//;. OThe impact o* empirical accuracy studies on time series analysis and *orecastingP, International $tatistical Review, /) ,:2, pp. @0/Q:A0. 3anke, C.E. and +.$. 4eitsch, .//;. ,usiness 3orecasting, Brentice 3all, Engle(ood 9li**s, NC. 3ansen, C.?. and 4.D. Nelson, @AA:. O<orecasting and recombining time'series components by using neural net(orksP, 1ournal of the 'perational Research $ociety, -, ,:2, pp. :A1Q:.1. 3ill, T., !. 589onnor and -. 4emus, .//=. ONeural net(ork models *or time series *orecastsP, /anagement $cience, ,%, pp. .A0@Q.A/@. 3olt, 9.9., ./;1. O<orecasting seasonal and trends by e)ponentially (eighted mo#ing a#eragesP, 'ffice of Naval Research, /emorandum, No. -%. 3ornik, >., !. Stichcombe and 3. -hite, ./0/a. O!ultilayer *eed*or(ard net(orks are uni#ersal appro)imatorsP, Neural Networks, %, pp. :;/Q:==. 3ornik, >., !. Stichcombe and 3. -hite, ./0/b. O"ni#ersal appro)imation o* an unkno(n mapping and its deri#ati#es using multilayer *eed*or(ard net(orksP, Neural Networks, ), pp. ;;.Q;=A. >aashoek, C.<. and 3.>. ?an Di%k, @AA.. Neural Networks as 2conometric &ool, 4eport EI @AA.QA;, Econometric Inst. Erasmus "ni#ersity 4otterdam. !akridakis, S. and S.9. -heel(right, ./01. &he 5andbook of 3orecasting( . /anager6s 7uide, @nd Edition, Cohn -iley R Sons Inc., Ne( &ork. Nam, >. and T. Schae*er, .//;. O<orecasting international airline passenger tra**ic using neural net(orksP, ogistics and &ransportation Review, )1 ,:2, pp. @:/Q@;..

S"3+4T5N5 +ND S"6+N+4

Nelson, !., T. 3ill, T. 4emus and !. 589onnor, .///. OTime series *orecasting using NNs: Should the data be deseasonaliGed *irstLP, 1ournal of 3orecasting, 10, pp. :;/Q:=1. Bersons, -.!., ././. OIndices o* business conditionsP, Review of 2conomics and $tatistics 1, pp. ;Q.A1. Bersons, -.!., ./@:. O9orrelation o* time seriesP, 1ournal of .merican $tatistical .ssociation, 10, pp. ;Q.A1. Suhartono, Subanar and S. $uritno, @AA;a. O+ 9omparati#e study o* *orecasting models *or trend and seasonal time series: Does comple) model al(ays yield better *orecast than simple modelsLP, 1urnal &eknik Industri( 1urnal 8eilmuan dan .plikasi &eknik Industri, ?ol. ., No. ., pp. @@':A. Suhartono, Subanar and S. 4eGeki, @AA;b. O<eed*or(ard Neural Net(orks !odel *or <orecasting Trend and Seasonal Time SeriesP, +roceedings of the 9st I/&0 7& Regional -onference on /athematics, $tatistics and &heir applications , !edan, North Sumatera, Indonesia. -ei, -.-.S., .//A. &ime $eries .nalysis( :nivariate and /ultivariate /ethods, +ddison'-esley Bublishing 9o., "S+. -hite, 3., .//A. O9onnectionist nonparametric regression: !ultilayer *eed *or(ard net(orks can learn arbitrary mappingP, Neural Networks, ), ;:;Q;;A. -inters, B.4., ./=A. O<orecasting Sales by E)ponentially -eighted !o#ing +#eragesP, /anagement $cience, /, pp. :@E':E@. Dhang, $., 6.E. Batu(o and !.&. 3u, .//0. O<orecasting (ith arti*icial neural net(orks: The state o* the artP, International 1ournal of 3orecasting, 1,, pp. :;Q=@.

)*