Econ 198 – Special Topics in Economics

Advanced Topics in Econometrics

Lecture 1.4: Nature of Time Series Data, Assumptions under Regression Analysis
Asymptotic Properties of OLS in Time Series
• Theorems on asymptotic properties that holds for
random sampling may not hold for random
processes, such as the time series.
• The key is whether the correlation between the
variables at different time periods tend to zero
quickly.
• Otherwise, that is, time series may have
substantial temporal correlation require special
attention in regression analysis.
Stationary and Weakly Dependent
Time Series
 Stationary and Weakly Dependent
Time Series
• These are key concepts that are needed
to apply the large approximations in
regression analysis with time series data.
 Stationary Time Series
A stationary time series process is one whose
probability distributions are stable over time in the
following sense: If we take any collection of random
variables in the sequence and then shift that
sequence ahead h time periods, the joint
probability distribution must remain unchanged.
 Stationary Time Series
Definition:
The stochastic process 𝑥𝑡 ∶ 𝑡 = 1,2, … is stationary
if for every collection of time indices 1 ≤ 𝑡1 < 𝑡2 <
⋯ < 𝑡𝑚 , the joint distribution of 𝑥𝑡1 , 𝑥𝑡2 , … , 𝑥𝑡𝑚 is
the same as that of 𝑥𝑡1+ℎ , 𝑥𝑡2+ℎ , … , 𝑥𝑡𝑚 +ℎ for all
integers ℎ ≥ 1.
 Stationary Time Series
Implications:
• 𝑥𝑡 has the same distribution for all 𝑡 = 1,2, …
• In other words, the sequence 𝑥𝑡 ∶ 𝑡 = 1,2, … is
identically distributed. But stationarity requires
even more… including subsets of the sequence
• Note: no restrictions yet for the relationship
between 𝑥𝑡 and 𝑥𝑡+ℎ .
 Stationary Time Series
Implications:
• If a stochastic process is not stationarity, then is
nonstationary process.
 Stationary Time Series
Definition:
The stochastic process 𝑥𝑡 ∶ 𝑡 = 1,2, … is covariance
stationary if
(i) 𝐸 𝑥𝑡 is constant
(ii) 𝑉𝑎𝑟 𝑥𝑡 is constant and
(iii) for any 𝑡, ℎ ≥ 1, 𝐶𝑜𝑣(𝑥𝑡 , 𝑥𝑡+ℎ ) depends only
on ℎ and not on 𝑡
 Stationary Time Series
Implications:
• Thus, covariance stationarity is a weaker form of
stationarity and requires only that the mean and
variance are constant across time, and the
covariance just depends on the distance across
time.
• To emphasize that stationarity is a stronger
requirement than covariance stationarity, the
former is referred to as strict stationarity.
 Stationary Time Series
How is stationarity used in time series
econometrics?

Technical level: Stationarity simplifies statements of

the law of large numbers and the central limit
theorem
 Stationary Time Series
How is stationarity used in time series
econometrics?

Practical level: If we want to understand the

relationship between two or more variables using
regression analysis, we need to assume some sort
of stability over time.
 Stationary Time Series
How is stationarity used in time series
econometrics?

Practical level: If we allow the relationship between

two variables (say, 𝑦𝑡 and 𝑥𝑡 ) to change arbitrarily in
each time period, then we cannot hope to learn
much about how a change in one variable affects
the other variable if we only have access to a single
time series realization.
 Weakly Dependent Time Series
Definition:
A stationary time series 𝑥𝑡 ∶ 𝑡 = 1,2, … is weakly
dependent if 𝑥𝑡 and 𝑥𝑡+ℎ are “almost independent”
as ℎ increases.
Weakly Dependent Time Series
Implications:
• If for a covariance stationary process
𝐶𝑜𝑟𝑟(𝑥𝑡 , 𝑥𝑡+ℎ ) → 0 “sufficiently quickly” as ℎ →
∞, we’ll say this covariance stationary process is
weakly dependent. (asymptotically uncorrelated)
• In other words, as the variables get farther apart
in time, the correlation between them becomes
smaller and smaller.
Weakly Dependent Time Series
Implications:
• Is weak dependence really important for
regression analysis? Essentially, it replaces the
assumption of random sampling in implying that
the law of large numbers (LLN) and the central
limit theorem (CLT) hold.
Weakly Dependent Time Series
Implications:
• If for a covariance stationary process
𝐶𝑜𝑟𝑟(𝑥𝑡 , 𝑥𝑡+ℎ ) → 0 “sufficiently quickly” as ℎ →
∞, we’ll say this covariance stationary process is
weakly dependent. (asymptotically uncorrelated)
• In other words, as the variables get farther apart
in time, the correlation between them becomes
smaller and smaller.
Weakly Dependent Time Series
Example 1: Moving average process of order one
[MA(1)]:
𝑥𝑡 = 𝛼1 𝑒𝑡−1 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
where 𝑒𝑡 ∶ 𝑡 = 1,2, … is an independent and
identically distributed sequence with mean 0 and
2
variance 𝜎𝑒 .
Weakly Dependent Time Series
Example 1: Moving average process of order one
[MA(1)]:
𝑥𝑡 = 𝛼1 𝑒𝑡−1 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
Is this stochastic process stationary?
Is it covariance stationary?
Is it weakly dependent?
Weakly Dependent Time Series
Example 2: Autoregressive process of order one
[AR(1)]:
𝑦𝑡 = 𝜌1 𝑦𝑡−1 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
where 𝑒𝑡 ∶ 𝑡 = 1,2, … is an independent and
identically distributed sequence with mean 0 and
2
variance 𝜎𝑒 . We also assume that 𝐸 𝑦0 = 0.
Weakly Dependent Time Series
Example 2: Autoregressive process of order one
[AR(1)]:
𝑦𝑡 = 𝜌1 𝑦𝑡−1 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
Is this stochastic process stationary?
Is it covariance stationary?
Is it weakly dependent?
 Random Walk Model
A case where 𝜌1 = 1:
𝑦𝑡 = 𝑦𝑡−1 + 𝑒𝑡
• A random walk is defined as a process where the
current value of a variable is composed of the past
value plus an error term defined as a white noise (a
normal variable with zero mean and variance one).
 Random Walk Model
A case where 𝜌1 = 1:
𝑦𝑡 = 𝑦𝑡−1 + 𝑒𝑡
• In a random walk model, the series itself is not
random.
• However, its differences—the changes from one
period to the next—are random.
• This type of behavior is typical of stock price data.
 Random Walk Model
A case where 𝜌1 = 1:
𝑦𝑡 = 𝑦𝑡−1 + 𝑒𝑡

• In practical terms: If series follows

random walk, original series
offers little or no insights.
• It would be hard to forecast
future trends
Weakly Dependent Time Series
Example 3: A time series with trend:
𝑦𝑡 = 𝛼0 + 𝛼1 𝑡 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
where 𝑒𝑡 ∶ 𝑡 = 1,2, … is an independent and
identically distributed sequence with mean 0 and
2
variance 𝜎𝑒 . We also assume that 𝐸 𝑦0 = 0.
Weakly Dependent Time Series
Example 3: A time series with trend:
𝑦𝑡 = 𝛼0 + 𝛼1 𝑡 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
Is this stochastic process stationary?
Is it covariance stationary?
Is it weakly dependent?
Weakly Dependent Time Series
However, if we take the deviation from the mean,
𝑦𝑡 − 𝐸 𝑦𝑡
where 𝑦𝑡 = 𝛼0 + 𝛼1 𝑡 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
can be stationary time series process, weakly
dependent.
Weakly Dependent Time Series
The previous is an example of: Trend-
Stationary Process, that is, a series that is
stationary about its time trend as well as
weakly dependent.

Thus the random shocks to the process are

transitory and the process is ‘mean reverting’,
with the mean
Weakly Dependent Time Series
How about getting the first difference?
∆𝑦𝑡 = 𝑦𝑡 − 𝑦𝑡−1
where 𝑦𝑡 = 𝛼0 + 𝛼1 𝑡 + 𝑒𝑡
𝑡 = 1,2, … , 𝑛
Can this be a stationary time series process, and
weakly dependent?