**Reducing scatter plots to a coherent function via MSE.** Multiple regression testing is a method to derived which **weights** for a weighted average produces the minimum Mean Squared Error. How to perform MRT wasn’t covered in the text. Guess and check seems like a fine enough approach to begin with. Calculus if you want, sure. Regression testing is a technique in which a scatter plot of data can be reduced to a single function approximating the scatter. This is done using the Mean Squared Error ([[Mean Squared Error]]) technique. Whatever function produces the minimal MSE is the best function... and the size of the minimum MSE is an indicator of system variability (aka [[Noise]]). There are other forms of regression, aside from [[Linear Regression]]. And there are other forms of [[Model Fitting Norm]]s that can be used aside from [[Mean Squared Error]]. Remember that [[Statistical Summaries May Hide Truths]]. The Datasauras Rex can produce a line. # Model Failure ## Under Fitting If your model does not have enough internal parameters (or you simply use the wrong curve), its predictive power will suck. This is typical of an under-fit model. We call models that are under-fit "**high [[Bias]]**". **Example:** trying to model a parabola using a straight line. ## Over Fitting The opposite of under fitting, it's possible to give a model **too much freedom** to match the input data, thereby letting it match those inputs _directly_, which can make it perform **worse** when using it for predictions. This is a common problem with [[Machine Learning]], also. This happens when the number of internal parameters is too high with respect to the number of inputs. We call models that are over-fit "**high variance**". **Example:** Degrees 5 & 7 applied against roughly quadratic data: ![[Pasted image 20250330104233.png]] If you were to predict the value at input = 6 (or anything greater than 5) you'd get _very wrong_ answers due to over fitting. **** # More ## Source - [[Noise - A Flaw in Human Judgment]] - [[Wikipedia]] - [Mean squared error - Wikipedia](https://en.wikipedia.org/wiki/Mean_squared_error) ## Related - [[Mean Squared Error]] - [[Noise]]