r/statistics • u/mandemting03 • 1d ago

Question [Q] Moving Average model. Iterative process to figure out residuals & coefficients?

Edit: The guy in the video mentions something about "iterative convergence". I'm assuming he means how the residuals and coefficients converge to their true values after multiple iterations

I recently started learning about Moving Average models and came across this video where the guy does an iterative process until which he gets the correct residuals and coefficients but I can't for the life of me understand the theory behind why it works.

Basically, for the very first iteration he assumes the errors are the demeaned values. He then regresses them against the Y variables and ends up with coefficients. He then calculates new residuals from the 1st iterative model and uses them as the regressors for the next iteration. He repeats this until the residuals and coefficients barely change.

Why and how does this work?

The only thing I'm familiar with for the MA residuals process is Maximum Likelihood but that's not what he's doing here at all.

Thank you very much

5 Upvotes

78% Upvoted

5 comments sorted by

2

u/gyp_casino 1d ago

It actually *is* related to Maximum Likelihood, because MLE typically requires a numerical solver.

You start with a guess of the values, calculate a derivative numerically, and take steps (descend) in the direction towards the minimum.

There are many algorithms for doing so, but some famous ones are c("Nelder-Mead", "BFGS", "CG", "L-BFGS-B", "SANN", "Brent"). (These are the available options in R optim())

1

u/mandemting03 1d ago

From the way I understood it is that in MLE you try a certain set of parameters and compare the Maximum Likelihood to the next set of parameters and if the values are getting minimized then you're going in the right direction.

Here on the other hand, the guy just starts from demeaned values as errors, gets a model, calculates new errors and then plucks those into the next iteration, rinse and repeat and that's it.

1

u/purple_paramecium 1d ago

Yeah, and you have to get the “certain set of parameters” to “try” the first round from somewhere…. You can start with a stupid guess like 10 million. Or you can use a rule-of-thumb to pick a reasonable starting guess. That’s what the guy is doing.

1

u/mandemting03 1d ago

What I'm trying to understand is that in his case it seems like it's just guaranteed that each iteration will just get more accurate which isn't necessarily the case with MLE(that's why a machine trying out different parameters will undergo multiple corrections as it reaches the optimal parameters)

His way just works no matter what. At no point did his iterative models stray away from the optimal errors/coefficients.

1

u/purple_paramecium 5h ago

Iterative least squares has theoretical convergence guarantees.

https://mdav.ece.gatech.edu/ece-4803-fall2020/notes/06-notes-4803-f20.pdf