Skip to content
Back to exploration
Mathematical modeling / English

What are ARIMA Models

Aric LaBarr · YouTube · 5:07

Open original
READ & KEEP

The explanation, unpacked.

Saves 0

Reviewed learning material · Video analysis · English
Read the full overview

This 120-second lecture segment reviews AR and MA models, shows by recursive substitution that an AR(1) model can be written as an infinite moving-average model, and then introduces the general ARMA equation combining lagged observations and lagged errors. It closes by naming two approaches for choosing ARMA orders: plotting correlation patterns and using automatic selection techniques. This video segment introduces the concept of selecting appropriate autoregressive (AR) and moving average (MA) orders for ARIMA models. It outlines three automatic selection techniques—MINIC, SCAN, and ESACF—and explains their methodologies. The segment emphasizes that these techniques serve as starting points for model building rather than definitive solutions. This introductory video explains the components of ARIMA models for time series analysis. It covers automatic selection techniques using MINIC, SCAN, and ESACF grids to identify AR and MA orders. The video then defines the 'I' in ARIMA as integrated, explaining stationarity and how differencing is used to achieve it. Finally, it breaks down the ARIMA(p, d, q) notation and provides a concrete example of constructing an ARIMA(1,1,1) model.

Use the learning inspector for key ideas and moments, or open the reading tabs for the complete notes.

Chapters

0:00Title: What are ARIMA models?0:04Review of AR and MA definitions0:37Recursive expansion of AR(1)1:09AR(1) as MA(∞\infty ) and long vs short memory1:42General ARMA model definition1:57Intro to choosing p and q2:00Introduction to ARIMA Model Structure2:23Overview of Automatic Selection Techniques2:44Detailed Explanation of MINIC Method2:58Detailed Explanation of SCAN Method3:42Detailed Explanation of ESACF Method4:00Automatic Selection Techniques4:11Stationarity and Differencing4:35ARIMA Notation Explained4:45ARIMA(1,1,1) Example4:59Conclusion

Learning script

Generated from the video's visuals and explanation; not verbatim speech.

The clip opens with the title question “WHAT ARE ARIMA MODELS?” and immediately frames the topic as a continuation from earlier discussions of AR and MA models.

A review slide defines the two building blocks. AR means forecasting from past observed values of the series, called lags. MA means forecasting from past errors, called error lags. The accompanying sales-versus-time chart visually anchors these ideas in a real time series.

The lecture then shifts to the relationship between the two. Starting from the AR(1) equation Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t, the speaker explains that because Yt−1Y_{t-1} itself depends on Yt−2Y_{t-2}, the current value inherits dependence farther and farther back in time. This recursive backward view is why AR models are described as long-memory models.

The algebra is continued on screen: after substitution, the model is rewritten as Yt=ω∗+ϕ2Yt−2+ϕet−1+etY_t = \omega^* + \phi^2 Y_{t-2} + \phi e_{t-1} + e_t, and then expanded further until it reaches Yt=ω∗+ϕtY1+ϕt−1e2+ϕt−2e3+⋯+etY_t = \omega^* + \phi^t Y_1 + \phi^{t-1} e_2 + \phi^{t-2} e_3 + \cdots + e_t. The important visual move is to ignore the initial-condition term and focus on the accumulated error terms.

That accumulated error piece is identified as a large moving-average component. If the recursion is not stopped at a finite starting observation, the slide summarizes the result as AR(1)=MA(∞)AR(1)=MA(\infty). In words, the video presents an AR model as an infinite MA model built from many short-memory shock effects.

The next slide interprets this conceptually: long memory in AR arises from collecting many short-memory MA effects. This motivates the practical step of combining AR and MA structures rather than relying on an extremely long pure AR or pure MA representation.

The general ARMA model is then written explicitly as Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \cdots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \cdots + \theta_q e_{t-q} + e_t. The speaker explains that ARMA models usually need fewer terms while still capturing the dynamics well.

Finally, the clip introduces the next problem: deciding how many pp’s and qq’s to use. Two methods are named on the slide—plotting correlation patterns and automatic selection techniques—but the explanation is cut off before either method is developed.

The video begins by addressing the challenge of determining the number of autoregressive (AR) and moving average (MA) terms, denoted as p and q respectively, in an ARIMA model.

Due to time constraints, the presenter opts for computational approaches over traditional methods like plotting autocorrelation and partial autocorrelation functions, which can become complex for intricate models.

Three primary automatic selection techniques are introduced: MINIC, SCAN, and ESACF. These methods are described as starting points for model building, encouraging further refinement through trial and error.

MINIC, or Minimum Information Criterion, involves constructing a grid of potential AR and MA orders, fitting models for each combination, and selecting the one with the lowest Bayesian Information Criterion (BIC).

SCAN, or Squared Canonical Correlations, analyzes the correlation matrix of the data and its lags, extracting eigenvalues to identify AR and MA orders that yield near-zero canonical correlations.

ESACF, the Extended Sample Autocorrelation Function, iteratively filters out AR terms until only MA components remain, ensuring that residual correlations are minimized.

The video begins by discussing automatic selection techniques for time series models, specifically mentioning MINIC, SCAN, and ESACF. Visual grids are used to illustrate how these techniques help identify the appropriate number of Autoregressive (AR) and Moving Average (MA) terms by looking for patterns like upper right triangles.

Next, the focus shifts to the 'I' in ARIMA, which stands for 'integrated'. This relates to the concept of stationarity in time series data. Stationarity is defined as the property where the data's distribution depends only on the time difference between observations, not their absolute position in time. To achieve stationarity, the video explains that differencing is typically applied to the data.

The presentation then summarizes the standard notation for ARIMA models: ARIMA(p, d, q). It clearly defines each parameter: 'p' represents the number of AR terms, 'd' is the number of first differences applied, and 'q' denotes the number of MA terms.

To solidify the concept, a specific example of an ARIMA(1,1,1) model is constructed. The process involves first taking a single difference (d=1d=1) of the original series YtY_t to create a stationary series WtW_t. Then, an ARMA(1,1) model is fitted to WtW_t, incorporating one autoregressive term (p=1p=1) and one moving average error term (q=1q=1).

The video concludes by reiterating that these steps and definitions encompass what ARIMA models are and how they can be selected.

Knowledge cards

01

AR models use past observations

The video defines an autoregressive model as one that forecasts a series from its own past values. These past values are called lags. This distinguishes AR from MA, which uses past errors instead.

02

MA models use past errors

A moving-average model is defined as forecasting from past error terms, called error lags. The clip emphasizes that these are shocks to the series, not previous observed values.

03

AR models are long-memory models

Because an AR model can be expanded recursively backward in time, earlier observations continue to influence the present. The video uses this recursive dependence to explain the phrase long memory.

Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t
04

Recursive expansion of AR(1)

Starting from Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t, substituting backward produces expressions involving Yt−2Y_{t-2}, then earlier lags, and eventually Y1Y_1 plus a weighted sum of past errors. The slide highlights the error-sum part as the key moving-average component.

Yt=ω∗+ϕtY1+ϕt−1e2+ϕt−2e3+⋯+etY_t = \omega^* + \phi^t Y_1 + \phi^{t-1} e_2 + \phi^{t-2} e_3 + \cdots + e_t
05

AR(1) = MA(∞\infty )

If the recursion is allowed to continue indefinitely rather than stopping at a first observation, the AR(1) model becomes an infinite moving-average model in past errors. This is the central equivalence stated in the clip.

AR(1)=MA(∞)AR(1) = MA(\infty)
06

Long memory as accumulated short-memory effects

The video interprets the AR structure as a collection of many short-memory MA effects. Each lagged error contributes a small piece, and together they generate the longer-lived dependence associated with AR models.

07

General ARMA model

ARMA combines autoregressive terms in past observations with moving-average terms in past errors. The stated advantage is parsimony: one can often replace very large pure AR or pure MA models with a smaller mixed ARMA model.

Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \cdots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \cdots + \theta_q e_{t-q} + e_t
08

Choosing p and q in ARMA

After defining ARMA, the clip introduces the next task: determining how many AR and MA terms to include. Two approaches are named: plotting correlation patterns and using automatic selection techniques.

09

ARIMA Model Definition

The ARIMA model integrates autoregressive (AR) and moving average (MA) components to forecast time series data. Its general form includes constants, AR terms, MA terms, and a white noise error term.

Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t
10

Automatic Selection Techniques

MINIC, SCAN, and ESACF are computational methods designed to assist in choosing suitable AR and MA orders for ARIMA models. They serve as initial guides rather than conclusive tools.

11

MINIC Method

MINIC evaluates various AR and MA order combinations via a grid search and selects the model with the lowest Bayesian Information Criterion (BIC), providing a systematic approach to model selection.

12

SCAN Method

SCAN examines the correlation matrix of the data and its lags, focusing on eigenvalues (squared canonical correlations) to pinpoint AR and MA orders that result in minimal correlations.

13

ESACF Method

ESACF progressively eliminates AR terms until only MA components persist, ensuring that the remaining autocorrelations are effectively reduced, thus identifying optimal ARIMA orders.

14

Automatic Model Selection Grids

Techniques like MINIC, SCAN, and ESACF use grids mapping AR and MA orders to suggest potential model structures. Analysts look for specific patterns, such as upper right triangles, to identify candidate ARMA(p,q) models.

15

Stationarity in Time Series

A time series is stationary if its statistical properties, like mean and variance, depend only on the time lag between observations, not the specific time point. This is crucial for many forecasting models.

16

Differencing for Stationarity

Differencing is a transformation applied to non-stationary time series to make them stationary. It involves calculating the difference between consecutive observations (e.g., Yt−Yt−1Y_t - Y_{t-1}).

Wt=Yt−Yt−1W_t = Y_t - Y_{t-1}
17

ARIMA(p, d, q) Notation

ARIMA models are parameterized by three integers: p (number of AR terms), d (degree of differencing), and q (number of MA terms).

ARIMA(p,d,q)\text{ARIMA}(p, d, q)
18

Example: ARIMA(1,1,1) Model

An ARIMA(1,1,1) model first differences the original series (d=1d=1) and then fits an ARMA(1,1) model to the differenced series, including one AR lag (p=1p=1) and one MA lag (q=1q=1).

Yt−Yt−1=WtWt=ω+ϕ1Wt−1+θ1et−1+etY_t - Y_{t-1} = W_t \\ W_t = \omega + \phi_1 W_{t-1} + \theta_1 e_{t-1} + e_t

Detailed learning notes

Explore conditions, steps and evidence. Supplementary explanations are labeled separately from content shown in the video.

Symbols · 22

YtY_t

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The slide shows the dependent variable as YtY_t in the AR(1) recursion and later in the general ARMA equation.

Symbol

YtY_t

Meaning

Time-series observation at time tt.

Domain

Observed series values used in autoregressive terms.

ete_t

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The slide shows ete_t, et−1e_{t-1}, et−2e_{t-2}, and later et−qe_{t-q} in the ARMA equation.

Symbol

ete_t

Meaning

Error term (innovation/shock) at time tt.

Domain

Past errors used in moving-average terms.

ϕ\phi, ϕ1\phi_1,…\ldots,ϕp\phi_p

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The AR(1) recursion uses ϕ\phi and ϕ2\phi^2, and the general ARMA formula uses ϕ1,…,ϕp\phi_1,\ldots,\phi_p.

Uncertainties
  1. The video does not explicitly state stationarity restrictions on ϕ\phi.

  2. The starred intercept ω∗\omega^* is shown but not defined verbally in this clip.

Symbol

ϕ\phi, ϕ1\phi_1,…\ldots,ϕp\phi_p

Meaning

Autoregressive coefficients multiplying lagged observations.

Domain

Parameters of AR terms.

θ1\theta_1,…\ldots,θq\theta_q

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The ARMA slide displays θ1et−1+⋯+θqet−q\theta_1 e_{t-1}+\cdots+\theta_q e_{t-q}.

Uncertainties
  1. The video does not explicitly state invertibility restrictions on θ\theta.

  2. No separate verbal definition of θi\theta_i is given in this clip.

Symbol

θ1\theta_1,…\ldots,θq\theta_q

Meaning

Moving-average coefficients multiplying lagged errors.

Domain

Parameters of MA terms.

ω\omega,\ ω∗\omega^*

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker does not define ω\omega or ω∗\omega^* in words during this clip.

Uncertainties
  1. The exact relationship between ω\omega and ω∗\omega^* is not explained in the clip.

  2. Whether ω∗\omega^* equals ω/(1−ϕ)\omega/(1-\phi) is not stated here.

Symbol

ω\omega,\ ω∗\omega^*

Meaning

Constant/intercept term in the model equations.

Domain

Additive intercept in AR and ARMA representations.

p,\ q

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says ARMA models combine AR and MA terms and typically have fewer terms.

Uncertainties
  1. The clip does not formally define pp and qq as integers beyond their appearance in the formula.

Symbol

p,\ q

Meaning

Order parameters: pp for autoregressive lags and qq for moving-average error lags.

Domain

Model orders in ARMA notation.

YtY_t

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

YtY_t

Meaning

Time series variable at time t

Domain

Real numbers

ω\omega

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

ω\omega

Meaning

Constant term in the ARIMA model

Domain

Real numbers

ϕi\phi_i

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

ϕi\phi_i

Meaning

Coefficients of the autoregressive (AR) terms

Domain

Real numbers

θj\theta_j

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

θj\theta_j

Meaning

Coefficients of the moving average (MA) terms

Domain

Real numbers

ete_t

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

ete_t

Meaning

White noise error term at time t

Domain

Real numbers

p

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Symbol

p

Meaning

Order of the autoregressive (AR) part

Domain

Non-negative integers

Knowledge points · 9

Definition of AR models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says AR models forecast a series based solely on past values of the series, called lags.

  2. Diagram
    Observation

    Slide text reads: 'AR – forecast a series based solely on the past values in the series – called lags.'

Definition
Explanation

An autoregressive (AR) model forecasts a time series using only its own past observed values, referred to as lags.

Formula
Conditions
  1. Forecasting a time series

  2. Uses past values of the same series

Definition of MA models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says MA models forecast a series based on past values of the errors in the series, called error lags.

  2. Diagram
    Observation

    Slide text reads: 'MA – forecast a series based solely on the past errors in the series – called error lags.'

Definition
Explanation

A moving-average (MA) model forecasts a time series using past error terms (shocks), called error lags.

Formula
Conditions
  1. Forecasting a time series

  2. Uses past errors rather than past observations

Long-memory interpretation of AR models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says AR models are called long memory models because one can recursively look back over time and still see effects from lags.

  2. Formula
    Observation

    The slide shows Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t and then Yt=ω∗+ϕ2Yt−2+ϕet−1+etY_t = \omega^* + \phi^2 Y_{t-2} + \phi e_{t-1} + e_t.

Uncertainties
  1. The clip does not state the convergence condition needed for an infinite backward expansion.

Definition
Explanation

AR models are described as long-memory models because the effect of earlier observations can be traced backward recursively through lagged dependence.

Formula
Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t
Conditions
  1. Applies to autoregressive time-series models

  2. Assumes recursive substitution backward in time

Prerequisites
  1. Definition of AR models

Definition of ARMA models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says that instead of very large AR or MA models, one can combine them into ARMA models that typically have fewer terms but model just as well.

  2. Formula
    Observation

    The slide title is 'Autoregressive Moving Average Models (ARMA)' and the text says 'Have both AR and MA terms together in the model.'

Uncertainties
  1. The claim that ARMA models 'typically have fewer terms but model just as well' is presented qualitatively, without proof in this clip.

Definition
Explanation

An ARMA model combines autoregressive terms in past observations with moving-average terms in past errors within a single equation.

Formula
Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \cdots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \cdots + \theta_q e_{t-q} + e_t
Conditions
  1. Time-series modeling

  2. Includes both lagged observations and lagged errors

Prerequisites
  1. Definition of AR models
  2. Definition of MA models

Introduction to ARMA model selection techniques

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    The final slide lists 'Plotting patterns in correlation' and 'Automatic selection techniques' under 'Couple of different techniques to selecting models'.

  2. Audio
    Observation

    The speaker begins to transition toward how to determine model order, but the sentence is cut off before explanation.

Uncertainties
  1. The clip ends before any method is explained in detail.

  2. No specific correlation plots or selection criteria are shown yet.

Method
Explanation

The video introduces two ways to choose model order: plotting correlation patterns and using automatic selection techniques.

Formula
Conditions
  1. Used when deciding how many AR and MA terms to include

Prerequisites
  1. Definition of ARMA models

ARIMA Model Definition

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

Definition
Explanation

The ARIMA model combines autoregressive (AR) and moving average (MA) terms to model time series data. The general form is given by the equation shown.

Formula
Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t
Conditions
  1. p and q are non-negative integers

  2. ete_t is white noise

Automatic Selection Techniques

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Three common techniques are called the MINIC, SCAN, and ESACF.

Method
Explanation

MINIC, SCAN, and ESACF are computational methods used to select appropriate AR and MA orders for an ARIMA model.

Formula
Conditions
  1. Used as starting points for model building

Prerequisites
  1. ARIMA Model Definition

Stationarity Definition

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Slide text states: 'Distribution depends only on difference in time, not location in time.'

  2. Audio
    Observation

    Speaker explains that stationarity means the distribution of your data depends only on differences in time, not location in time.

Definition
Explanation

In the context of time series, stationarity means that the statistical distribution of the data depends only on the difference in time between observations, rather than their absolute location in time.

Conditions
  1. Applies to time series data.

Differencing for Stationarity

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Slide text states: 'Essentially, you need to make your data stationary (typically done through differencing).'

  2. Audio
    Observation

    Speaker says, 'Typically, we take differences in our data to make our data stationary.'

Method
Explanation

Differencing is a technique used to transform a non-stationary time series into a stationary one by computing the differences between consecutive observations.

Conditions
  1. Used when data exhibits non-stationarity.

Prerequisites
  1. Stationarity Definition
Claims and conditions · 3

AR(1) can be represented as MA(∞\infty )

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says that if time is not stopped, an AR model is an infinite moving average model.

  2. Formula
    Observation

    The slide title changes to 'AR(1) = MA(∞\infty )' and boxes the error-only part of the expanded expression.

Uncertainties
  1. The clip does not state the formal condition ∣ϕ∣<1|\phi|<1 required for the infinite MA representation.

Proposition
Statement

By recursively substituting lagged YY terms backward, an AR(1) model can be rewritten as an infinite moving-average representation in past errors.

Hypotheses
  1. Start from an AR(1) equation of the form Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t.

  2. Continue the recursion far enough back in time.

  3. The clip leaves unstated the usual convergence/stationarity condition needed for the infinite limit.

Quantifiers

For the AR(1) model shown, repeated backward substitution yields an expression involving infinitely many lagged error terms if time is not stopped.

Long-memory AR effects arise from accumulated short-memory MA effects

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says AR and MA models are essentially opposites and that collecting short-memory MA effects makes the long-memory AR effect.

  2. Diagram
    Observation

    The slide title reads 'Long Memory and Short Memory?' and the bottom text says 'Collection of a bunch of short memory effects!'

Uncertainties
  1. This is an intuitive characterization rather than a formal theorem statement in the clip.

Proposition
Statement

The video presents AR and MA models as related opposites: many short-memory MA effects combine to produce the long-memory behavior associated with AR models.

Hypotheses
  1. Use the recursive expansion of an AR model into lagged error terms.

  2. Interpret each lagged error contribution as a short-memory effect.

Quantifiers

Qualitative claim about the relationship between AR and MA representations.

Purpose of Automatic Selection Techniques

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    These techniques are just starting points though... They give you good models to start with that you can play around from there to potentially build better models.

Proposition
Statement

Automatic selection techniques provide initial models that can be refined through trial and error.

Hypotheses
  1. Applicable to ARIMA model building

Quantifiers

For all automatic selection techniques

Derivations and proofs · 5

Recursive expansion of AR(1) into an MA form

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The narrator explains looking back recursively until the first observation Y1Y_1, then focusing on the right-hand side as a large moving-average term.

Uncertainties
  1. The algebraic derivation of ω∗\omega^* from ω\omega is skipped on screen.

  2. The passage from finite tt to the infinite MA representation is asserted verbally rather than proved rigorously.

  3. The clip does not mention the stationarity condition ∣ϕ∣<1|\phi|<1.

Proof
Steps
  1. Expression
    Yt=ω+ϕYt−1+etY_t = \omega + \phi Y_{t-1} + e_t
    Explanation

    Begin with the AR(1) model shown on the slide.

    Justification

    Given model equation.

    Shown in the video
  2. Expression
    Yt=ω∗+ϕ2Yt−2+ϕet−1+etY_t = \omega^* + \phi^2 Y_{t-2} + \phi e_{t-1} + e_t
    Explanation

    Substitute the lagged dependence backward one more step so that YtY_t is expressed using Yt−2Y_{t-2} and earlier errors.

    Justification

    Recursive substitution of the AR relation into itself.

    Shown in the video
  3. Expression
    ⋮\vdots
    Explanation

    Continue the same backward substitution repeatedly.

    Justification

    Iterating the recursion further back in time.

    Shown in the video
  4. Expression
    Yt=ω∗+ϕtY1+ϕt−1e2+ϕt−2e3+⋯+etY_t = \omega^* + \phi^t Y_1 + \phi^{t-1} e_2 + \phi^{t-2} e_3 + \cdots + e_t
    Explanation

    After expanding all the way back to the first observation, the expression contains a term in Y1Y_1 plus a weighted sum of past errors.

    Justification

    Finite backward expansion to the initial observation.

    Shown in the video
  5. Expression
    ϕt−1e2+ϕt−2e3+⋯+et\phi^{t-1} e_2 + \phi^{t-2} e_3 + \cdots + e_t
    Explanation

    The boxed portion isolates the accumulated error terms, which the speaker identifies as a large moving-average component.

    Justification

    Visual emphasis on the error-only part of the expanded equation.

    Shown in the video
  6. Expression
    AR(1)=MA(∞)AR(1) = MA(\infty)
    Explanation

    If the recursion is not stopped at a finite initial observation, the error expansion continues indefinitely, giving an infinite moving-average representation.

    Justification

    Verbal conclusion by the speaker plus slide title change.

    Shown in the video
Conclusion

The AR(1) model can be rewritten as an infinite moving-average model in past errors, motivating the slide statement AR(1)=MA(∞\infty ).

MINIC Method Derivation

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The MINIC... builds many combinations of models across a grid search of AR and MA terms. It then finds the model with the lowest Bayesian information criterion.

Intuitive argument
Steps
  1. Expression
    Explanation

    Construct a grid of possible AR and MA orders.

    Justification

    Grid search approach

    Shown in the video
  2. Expression
    Explanation

    Fit ARIMA models for each combination in the grid.

    Justification

    Model fitting process

    Shown in the video
  3. Expression
    Explanation

    Select the model with the lowest Bayesian Information Criterion (BIC).

    Justification

    Optimization criterion

    Shown in the video
Conclusion

MINIC selects the best ARIMA model based on BIC.

SCAN Method Derivation

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The SCAN... Look at the correlation matrix of your data. Then we're going to compare that with its lags. Then we're going to look at the eigenvalues from this correlation matrix... We want to find the combination of AR and MA terms that produce canonical correlations of approximately zero.

Intuitive argument
Steps
  1. Expression
    Explanation

    Compute the correlation matrix of the time series data.

    Justification

    Correlation analysis

    Shown in the video
  2. Expression
    Explanation

    Compare the correlation matrix with its lagged versions.

    Justification

    Lag comparison

    Shown in the video
  3. Expression
    Explanation

    Extract eigenvalues (squared canonical correlations) from the comparison.

    Justification

    Eigenvalue decomposition

    Shown in the video
  4. Expression
    Explanation

    Identify AR and MA orders where canonical correlations approach zero.

    Justification

    Zero-correlation criterion

    Shown in the video
Conclusion

SCAN identifies suitable ARIMA models by minimizing canonical correlations.

ESACF Method Derivation

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The ESACF... We know that the AR and MA terms are related. Essentially, we're filtering out AR terms until all we have left is the MA piece. If we filter and there's still AR correlations, then we haven't filtered enough ARs out.

Intuitive argument
Steps
  1. Expression
    Explanation

    Filter out AR terms sequentially.

    Justification

    Sequential filtering

    Shown in the video
  2. Expression
    Explanation

    Check remaining autocorrelations after each filter step.

    Justification

    Residual correlation check

    Shown in the video
  3. Expression
    Explanation

    Continue filtering until only MA components remain.

    Justification

    Termination condition

    Shown in the video
Conclusion

ESACF determines ARIMA orders by isolating MA components through iterative AR filtering.

Derivation of ARIMA(1,1,1) Model

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Slide displays ARIMA(1,1,1) with equations Yt−Yt−1=WtY_t - Y_{t-1} = W_t and Wt=ω+ϕ1Wt−1+θ1et−1+etW_t = \omega + \phi_1 W_{t-1} + \theta_1 e_{t-1} + e_t.

  2. Audio
    Observation

    Speaker explains that for an ARIMA(1,1,1), we first take a difference (d=1d=1) to go from Y to W, then have a lag of W(p=1)W (p=1), and a lag of error (q=1q=1).

Proof
Steps
  1. Expression
    Yt−Yt−1=WtY_t - Y_{t-1} = W_t
    Explanation

    Apply first-order differencing (d=1d=1) to the original time series YtY_t to obtain the stationary series WtW_t.

    Justification

    Definition of differencing for stationarity.

    Shown in the video
  2. Expression
    Wt=ω+ϕ1Wt−1+θ1et−1+etW_t = \omega + \phi_1 W_{t-1} + \theta_1 e_{t-1} + e_t
    Explanation

    Model the differenced series WtW_t using an ARMA(1,1) process, incorporating one AR term (p=1p=1) and one MA term (q=1q=1).

    Justification

    Definition of ARMA(p,q) model applied to the differenced series.

    Shown in the video
Conclusion

An ARIMA(1,1,1) model consists of taking a first difference of the original series and then fitting an ARMA(1,1) model to the resulting differenced series.

Worked examples · 2

SCAN Example Models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Here's one at an AR3 MA0 model. Here is another at an AR2 MA2 model. And here is yet another at an AR0 MA3 model.

Problem

Identify valid ARIMA models using SCAN method.

Given
  1. Correlation matrix and its lags

Goal

Find AR and MA orders with near-zero canonical correlations.

Steps
  1. Expression
    Explanation

    Examine correlation matrices for patterns.

    Justification

    Visual inspection

    Shown in the video
  2. Expression
    Explanation

    Identify rectangles indicating low canonical correlations.

    Justification

    Pattern recognition

    Shown in the video
Answer

Valid models include AR(3,0), AR(2,2), and AR(0,3).

Verification

Models show minimal canonical correlations.

ARIMA(1,1,1) Example

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    Slide explicitly shows the title 'ARIMA(1,1,1)' and the corresponding equations.

  2. Audio
    Observation

    Speaker says, 'So if we had an ARIMA 1 1 1, we have the following...'

Problem

Construct the mathematical representation of an ARIMA(1,1,1) model.

Given
  1. p=1p=1 (one AR term)

  2. d=1d=1 (one first difference)

  3. q=1q=1 (one MA term)

Goal

Write the equations defining the ARIMA(1,1,1) process.

Steps
  1. Expression
    Yt−Yt−1=WtY_t - Y_{t-1} = W_t
    Explanation

    Take the first difference of the original series YtY_t.

    Justification

    d=1d=1 indicates one order of differencing.

    Shown in the video
  2. Expression
    Wt=ω+ϕ1Wt−1+θ1et−1+etW_t = \omega + \phi_1 W_{t-1} + \theta_1 e_{t-1} + e_t
    Explanation

    Write the ARMA(1,1) equation for the differenced series WtW_t.

    Justification

    p=1p=1 and q=1q=1 indicate one autoregressive term and one moving average term.

    Shown in the video
Answer

The ARIMA(1,1,1) model is defined by the system: Yt−Yt−1=WtY_t - Y_{t-1} = W_t and Wt=ω+ϕ1Wt−1+θ1et−1+etW_t = \omega + \phi_1 W_{t-1} + \theta_1 e_{t-1} + e_t.

Verification

Matches the visual equations and audio explanation provided in the video.

Visual events · 6

Sales time-series illustration for AR and MA review

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    A slide titled 'Remember AR and MA Models?' shows a time-series chart with y-axis 'Sales ($ Millions)' and x-axis dates from 1/7/20051/7/2005 to 5/19/20055/19/2005.

  2. Animation
    Observation

    Around 21s and again around 28s, orange circles highlight points on the chart while the speaker reviews AR and MA definitions.

Uncertainties
  1. The chart illustrates lag/error concepts visually, but the exact numerical values of the highlighted points are not discussed verbally.

Objects
  1. Line chart of sales over time

  2. Orange highlight circles

  3. Bullet text defining AR and MA

Changes
  1. Static chart remains on screen while orange circles appear around selected points.

  2. Slide focus shifts from AR bullet to MA bullet.

Invariants
  1. The axes remain 'Sales ($ Millions)' versus dates.

  2. The textual definitions of AR and MA stay visible.

Interpretation

The visual supports the verbal distinction between using past observations (lags) and past errors (error lags) in forecasting.

Slide progression showing recursive AR expansion

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    Slides progress from 'Relationship Between AR and MA' to 'Keep Doing Math...' to 'AR(1) = MA(∞\infty )' to 'Long Memory and Short Memory?' while retaining the same recursive equations.

  2. Animation
    Observation

    An orange curved arrow links the first AR equation to the substituted second equation; later an orange box highlights the error-sum portion.

Uncertainties
  1. The intermediate algebra producing ω∗\omega^* is not shown step by step.

Objects
  1. AR(1) equation

  2. Substituted equation with Yt−2Y_{t-2}

  3. Ellipsis line

  4. Expanded equation ending in Y1Y_1 and error terms

  5. Orange arrow and orange box

Changes
  1. Equations accumulate downward as the recursion is extended.

  2. Title changes from relationship framing to 'Keep Doing Math...', then to 'AR(1) = MA(∞\infty )', then to 'Long Memory and Short Memory?'.

  3. The error-only tail is boxed for emphasis.

Invariants
  1. The core model remains an AR(1) equation being rewritten.

  2. The left-hand side stays YtY_t throughout.

Interpretation

The animation visually demonstrates that repeated substitution turns lagged observations into a growing weighted sum of past errors.

Selection Technique Grids

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    Three grids labeled MINIC, SCAN, ESACF showing AR vs MA orders.

Objects
  1. MINIC grid

  2. SCAN grid

  3. ESACF grid

Changes
  1. Highlighting of specific cells in grids

Invariants
  1. Grid structure remains constant

Interpretation

Each grid represents a different method for evaluating ARIMA model orders.

Automatic Selection Techniques Grids

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    Three 5x5 grids labeled MINIC, SCAN, and ESACF are shown, with axes labeled AR (0-4) and MA (0-4). Different cells are shaded to indicate possible model orders.

Objects
  1. MINIC grid

  2. SCAN grid

  3. ESACF grid

  4. AR axis

  5. MA axis

  6. Shaded cells

Changes
  1. Slide transitions from selection grids to the stationarity slide.

Invariants
  1. Axes remain AR (0-4) and MA (0-4) across all three grids.

Interpretation

The shaded regions in each grid represent the combinations of AR and MA orders suggested by the respective automatic selection technique. The speaker notes looking for upper right triangles and their points to identify models like ARMA(1,1), ARMA(2,2), etc.

ARIMA Notation Breakdown

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    Slide displays 'ARIMA(p, d, q)' with arrows pointing to definitions for p, d, and q.

Objects
  1. ARIMA(p, d, q) text

  2. Arrows

  3. Labels for p, d, q

Changes
  1. Slide transitions from stationarity to ARIMA notation.

Interpretation

Visually decomposes the ARIMA acronym, linking each parameter (p, d, q) to its specific meaning in the model structure.

ARIMA(1,1,1) Equations Visualized

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    Slide displays 'ARIMA(1,1,1)' with two equations below it, connected by arrows indicating the flow from YtY_t to WtW_t and then the ARMA equation.

Objects
  1. ARIMA(1,1,1) text

  2. Differencing equation

  3. ARMA equation

  4. Connecting arrows

Changes
  1. Slide transitions from general notation to specific example.

Interpretation

Illustrates the step-by-step construction of an ARIMA(1,1,1) model, showing how differencing creates a new series which is then modeled using ARMA terms.

Misconceptions · 3

Do not confuse AR lags with MA error lags

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker explicitly contrasts AR models, which use past values called lags, with MA models, which use past errors called error lags.

  2. Diagram
    Observation

    The slide separates the two definitions in parallel bullets.

Misconception

One might think AR and MA models differ only in name, or that both use the same kind of past information.

Clarification

AR models use past observed values of the series; MA models use past error terms (shocks) of the series.

Long memory in AR does not mean a single direct long lag

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says AR models are long-memory models and then explains that they can be viewed as collections of short-memory MA effects.

  2. Diagram
    Observation

    The slide title 'Long Memory and Short Memory?' and the boxed error sum support this interpretation.

Uncertainties
  1. The clip gives intuition rather than a formal memory-decay analysis.

Misconception

One might think 'long memory' means the model only depends on one distant lag.

Clarification

Here long memory means the recursive AR structure lets many past shocks accumulate, producing an effectively infinite MA representation.

Misconception about Automatic Selection

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    These techniques are just starting points though... They give you good models to start with that you can play around from there to potentially build better models.

Misconception

Automatic selection techniques provide final optimal models.

Clarification

They offer initial models that require further refinement.

Concept relations · 7

Definition of AR models → Definition of ARMA models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says ARMA models combine AR and MA terms together.

  2. Formula
    Observation

    The ARMA equation contains both ϕiYt−i\phi_i Y_{t-i} and θjet−j\theta_j e_{t-j} terms.

Contains
Explanation

The ARMA model includes autoregressive terms as one component of its general form.

Definition of MA models → Definition of ARMA models

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker says ARMA combines AR and MA models into one framework.

  2. Formula
    Observation

    The ARMA equation includes lagged error terms with coefficients θ1,…,θq\theta_1,\ldots,\theta_q.

Contains
Explanation

The ARMA model includes moving-average terms as the other component of its general form.

Long-memory interpretation of AR models → AR(1) can be represented as MA(∞\infty )

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The slide title becomes 'AR(1) = MA(∞\infty )' after the recursive expansion.

  2. Audio
    Observation

    The speaker states that an AR model is an infinite moving average model if time is not stopped.

Uncertainties
  1. The formal condition for equivalence is omitted in the clip.

Equivalent
Explanation

The long-memory AR representation is shown to be equivalent, under the video's recursive argument, to an infinite MA representation.

Definition of ARMA models → Introduction to ARMA model selection techniques

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    The slide 'Deciding How Many p's and q's...' appears directly after the ARMA definition slide.

  2. Audio
    Observation

    The speaker transitions from defining ARMA to asking how to determine the model order.

Uncertainties
  1. The actual selection procedures are not yet explained in this clip.

Application
Explanation

Once ARMA models are defined, the next issue is choosing the orders pp and qq.

ARIMA Model Definition → Automatic Selection Techniques

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Deciding how many p's and q's... Couple of different techniques to selecting models...

Application
Explanation

Automatic selection techniques apply to determining parameters in ARIMA models.

Differencing for Stationarity → Stationarity Definition

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Speaker connects making data stationary with taking differences.

  2. Formula
    Observation

    Slide text links 'make your data stationary' with '(typically done through differencing)'.

Application
Explanation

Differencing is a method applied to achieve stationarity in time series data.

Differencing for Stationarity → Derivation of ARIMA(1,1,1) Model

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    ARIMA(p,d,q) notation includes 'd' for first differences, and the ARIMA(1,1,1) example shows the differencing equation.

Proof dependency
Explanation

The derivation and understanding of the ARIMA model rely on the concept of differencing to handle non-stationary data.

Find an answer · 9

What is an ARMA model and what is its general equation?

Clear evidence
Shown in the video
Evidence
  1. Formula
    Observation

    The ARMA equation is displayed on screen.

  2. Audio
    Observation

    The speaker defines ARMA as combining AR and MA terms.

Knowledge points
  1. Definition of ARMA models

Why are AR models described as long-memory models?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    The speaker calls AR models long memory models and explains recursive backward dependence.

  2. Formula
    Observation

    The recursive expansion slides show YtY_t rewritten in terms of earlier YY and ee values.

Knowledge points
  1. Long-memory interpretation of AR models
  2. AR(1) can be represented as MA(∞\infty )

What is the difference between AR lags and MA error lags?

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    The slide separately defines AR and MA using lags versus error lags.

  2. Audio
    Observation

    The speaker contrasts past values with past errors.

Knowledge points
  1. Definition of AR models
  2. Definition of MA models

How do you decide how many AR and MA terms to include in an ARMA model?

Clear evidence
Shown in the video
Evidence
  1. Diagram
    Observation

    The final slide lists two techniques for selecting models.

  2. Audio
    Observation

    The speaker begins transitioning to how to determine model order before the clip ends.

Uncertainties
  1. Only the names of the techniques are present; no detailed method is shown.

Knowledge points
  1. Introduction to ARMA model selection techniques

How to choose p and q in ARIMA?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    How do I decide the number of AR and MA terms in an ARIMA model?

Knowledge points
  1. ARIMA Model Definition
  2. Automatic Selection Techniques

What does stationarity mean in time series analysis?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Speaker defines stationarity.

  2. Formula
    Observation

    Slide text defines stationarity.

Knowledge points
  1. Stationarity Definition

How do you make time series data stationary?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Speaker explains differencing to make data stationary.

  2. Formula
    Observation

    Slide text mentions differencing for stationarity.

Knowledge points
  1. Differencing for Stationarity

What do the parameters p, d, and q represent in an ARIMA model?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Speaker defines p, d, and q.

  2. Formula
    Observation

    Slide visually breaks down ARIMA(p,d,q).

Knowledge points
  1. p
  2. d
  3. q

Can you show me an example of an ARIMA(1,1,1) model and its equations?

Clear evidence
Shown in the video
Evidence
  1. Audio
    Observation

    Speaker walks through the ARIMA(1,1,1) example.

  2. Formula
    Observation

    Slide shows the equations for ARIMA(1,1,1).

Knowledge points
  1. ARIMA(1,1,1) Example
  2. Derivation of ARIMA(1,1,1) Model
Coverage and review notes

Covered · Title card 'WHAT ARE ARIMA MODELS?' with no mathematical content beyond topic introduction.

Covered · Review slide defines AR as using past values (lags) and MA as using past errors (error lags), with a sales time-series chart.

Covered · Slide 'Relationship Between AR and MA' and 'Keep Doing Math...' introduce recursive substitution of the AR(1) model.

Covered · Slide 'AR(1) = MA(∞\infty )' and 'Long Memory and Short Memory?' emphasize the infinite moving-average interpretation of AR.

Covered · General ARMA equation is shown and verbally motivated as a combination of AR and MA terms.

Covered · Final slide introduces two model-selection approaches: correlation plotting and automatic selection techniques.

Covered · Introduction to ARIMA model structure

Covered · Explanation of automatic selection techniques

Covered · Discussion of automatic selection techniques (MINIC, SCAN, ESACF) using grids to identify ARMA model orders.

Covered · Explanation of the 'I' in ARIMA, defining stationarity and the use of differencing to achieve it.

Covered · Breakdown of the standard ARIMA(p, d, q) notation and the meaning of each parameter.

Covered · Detailed example of constructing an ARIMA(1,1,1) model, showing the differencing step and the subsequent ARMA equation.

Covered · Closing remarks summarizing the video's content on ARIMA models and selection.

Explore the knowledge in this video

Open video knowledge graph →

  • ARIMA and differencing ExplanationAt 2:00
    Why this connection?

    Reviewed English material v1: card/concept_card "ARIMA Model Definition" starting at 2:00 — The ARIMA model integrates autoregressive (AR) and moving average (MA) components to forecast time series data. Its general form includes constants, AR terms, MA terms, and a white noise error term. Yt=ω+ϕ1Yt−1+⋯+ϕpYt−p+θ1et−1+⋯+θqet−q+etY_t = \omega + \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \theta_1 e_{t-1} + \dots + \theta_q e_{t-q} + e_t

  • Time series and exponential smoothing ExplanationAt 4:11
    Why this connection?

    Reviewed English material v1: card/concept_card "Stationarity in Time Series" starting at 4:11 — A time series is stationary if its statistical properties, like mean and variance, depend only on the time lag between observations, not the specific time point. This is crucial for many forecasting models.

Questions this video answers

Meet the concept

↗
Understand why

↗
Meet the concept

↗