**Model Regularization** refers to the act of applying [[Model Fitting Norm]]s to [[Regression Testing|regressive prediction models]] to prevent [[Regression Testing#Over Fitting|Over fitting]] input data. A model that has been trained via penalization functions is a _regularized_ model. Model regularization is effected by difference is scale, so it's good practice to normalize your data before regularizing the model. # Methods of Regularization ## Ridge Uses the internal parameters to tune the [[Mean Squared Error|MSE]] function. Includes a **hyper parameter** that determines its strength (0 = no effect, ∞ = just doing the mean). Ridge is a good default. ## Lasso Least Absolute Shrinkage and Selection Operator[^1] uses the absolute value of the internal parameters. This does not have a closed-form solution, unlike the above. This means "search" has to be used (i.e. guess & check). This also includes a **hyper parameter**. Lasso can be good when only a small number of features actually contribute to the output. ## Elastic Net Combines the above 2 methods. Includes two **hyper parameters** 1. How much the extra terms contribute, just like the above 2. What proportion of _lasso_ and _ridge_ are used - 0 = all Lasso - 0.5 = 50% each - 1 = all Ridge ## Early Stopping This is the method we used in our [[Machine Learning]] class. Take the data, separate it into a **training set** and a **testing set**. Train the regression on the training set and check the model's accuracy against the testing set. **** # More ## Source - Grad School [^1]: a definite [[Backronym]]