Lasso and Ridge
..............................................................................................................................................................
Lasso and Ridge are two types of regularization methods used in linear regression to prevent overfitting.
Lasso (Least Absolute Shrinkage and Selection Operator) is a type of regularization method that adds a penalty term to the loss function equal to the absolute value of the magnitude of the coefficients. The penalty term is controlled by a hyperparameter, called alpha, which determines the strength of the regularization. The formula for Lasso regression is:
Loss = Sum of Squared Residuals + alpha * Sum of absolute values of coefficients
The main idea behind Lasso is to shrink the coefficients of less important features to zero, effectively removing them from the model and achieving feature selection.
Ridge Regression, on the other hand, adds a penalty term to the loss function equal to the square of the magnitude of the coefficients. The formula for Ridge regression is:
Loss = Sum of Squared Residuals + alpha * Sum of squares of coefficients
The main idea behind Ridge is to shrink the coefficients of all features towards zero, but not to the extent that they are set to zero. This helps to prevent overfitting by reducing the complexity of the model.
Here's a simple example in Python to illustrate the use of Lasso and Ridge regression:
This will generate a scatter plot of the sample data and the predictions made by both Lasso and Ridge regression. You can see how the Lasso model is able to remove features that are less important, while the Ridge model shrinks all coefficients towards zero but does not eliminate any features.
Ridge Works well when there is a large number of features or multicollinearity
In simple terms, Lasso shrinks the less important feature's coefficient
to zero, making the solution sparse, while Ridge shrinks the
coefficients but doesn't make them zero.
Comments
Post a Comment