Notes in Week 2 - Multivariable regression

To Subscribe, use this Key

Status	Last Update	Fields
Published	10/15/2024	Robot localization uses {{c1::multiple beacons}} to predict a robot's distance to a reference point.
Published	10/15/2024	The formula for the univariate prediction model is: {{c1::\(f_{w,b}(x) = w x + b\)}}.
Published	10/15/2024	The formula for multiple linear regression is: {{c1::\(f_{w,b}(x) = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b\)}}.
Published	10/15/2024	In multiple linear regression, the feature vector is represented as: {{c1::\(\mathbf{x} = [x_1, x_2, \dots, x_n]\)}}.
Published	10/15/2024	The weight vector in multiple linear regression is: {{c1::\(\mathbf{w} = [w_1, w_2, \dots, w_n]\)}}.
Published	10/15/2024	The formula for non-vectorized multiple linear regression is: {{c1::\(f_{w,b}(x) = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b\)}}.
Published	10/15/2024	Vectorization of the multiple linear regression model is written as: {{c1::\(f_{w,b}(\mathbf{x}) = \mathbf{w} \cdot \mathbf{x} + b\)}}.
Published	10/15/2024	With vectorization, operations can be {{c1::parallelized}}.
Published	10/15/2024	In gradient descent, weights are updated using the formula: {{c1::\(w_j = w_j - \alpha \cdot d_j\)}}, where α is the learning rate.
Published	10/15/2024	Vectorized gradient descent updates weights as: {{c1::\(\mathbf{w} = \mathbf{w} - \alpha \cdot \mathbf{d}\)}}.
Published	10/15/2024	Feature scaling ensures that features have similar {{c1::scale ranges}} to improve gradient descent performance.
Published	10/15/2024	Mean normalization is a common feature scaling technique, with the formula: {{c1::\(x_{norm} = \frac{x - \text{mean}(x)}{\text{max}(x) - \text{min}(x)…
Published	10/15/2024	Z-score normalization (standardization) scales features using the formula: {{c1::\(x_{norm} = \frac{x - \text{mean}(x)}{\text{std}(x)}\)}}.
Published	10/15/2024	Polynomial regression extends linear regression by adding higher-degree terms: {{c1::\(f_{w,b}(x) = w_1 x + w_2 x^2 + \dots + w_n x^n + b\)}}.
Published	10/15/2024	Feature scaling is important for {{c1::polynomial regression}} because features raised to higher powers have different ranges.
Published	10/15/2024	In the robot localization example, the feature set includes {{c1::signal strengths from beacons}} and {{c2::LIDAR distance measurements}}.
Published	10/15/2024	The cost function for multiple linear regression is minimized using {{c1::gradient descent}}.
Published	10/15/2024	The learning curve shows how the cost function {{c1::decreases}} as the number of iterations in gradient descent increases.
Published	10/15/2024	The learning rate α determines the {{c1::step size}} in gradient descent.
Published	10/15/2024	Feature scaling improves the {{c1::convergence speed}} of gradient descent.
Published	10/15/2024	The formula for feature scaling using min-max normalization is: {{c1::\(x_{norm} = \frac{x - \text{min}(x)}{\text{max}(x) - \text{min}(x)}\)}}.
Published	10/15/2024	The formula for updating weights using vectorized gradient descent is: {{c1::\(\mathbf{w} = \mathbf{w} - \alpha \cdot \mathbf{d}\)}}, where α is the l…
Published	10/15/2024	{{c1::Positioning drift}} refers to errors in a robot's position estimation due to accumulated errors in measurements.
Published	10/15/2024	{{c1::Outliers}} are data points that deviate significantly from the other observations and can negatively affect the performance of linear regression…
Published	10/15/2024	A {{c1::regularization term}} is added to the cost function to penalize large weights and avoid overfitting.
Published	10/15/2024	The cost function for L2 regularization (Ridge regression) includes the regularization term: {{c1::\(J(w,b) = \frac{1}{2m} \sum (f_{w,b}(x^{(i)}) - y^…
Published	10/15/2024	The regularization parameter ({{c1::\(\lambda\)}}) controls the strength of the penalty in the regularized cost function.
Published	10/15/2024	Regularization can prevent {{c1::overfitting}} by discouraging overly complex models that fit the training data too closely.
Published	10/15/2024	When features have vastly different ranges, it can be helpful to apply {{c1::feature scaling}} before running linear regression.
Published	10/15/2024	In multivariate linear regression, the predicted output is a linear combination of the features and weights: {{c1::\(f_{w,b}(x) = w_1 x_1 + w_2 x_2 + …
Published	10/15/2024	To minimize the cost function for multivariate regression, we can use {{c1::gradient descent}}, which iteratively updates the weights.
Published	10/15/2024	The cost function for multivariate linear regression is: {{c1::\(J(w,b) = \frac{1}{2m} \sum_{i=1}^{m} (f_{w,b}(x^{(i)}) - y^{(i)})^2\)}}.
Published	10/15/2024	In linear regression, the model’s performance is often evaluated using {{c1::mean squared error}} (MSE).
Published	10/15/2024	The formula for {{c2::mean squared error}} is: {{c1::\(MSE = \frac{1}{m} \sum_{i=1}^{m} (f_{w,b}(x^{(i)}) - y^{(i)})^2\)}}.
Published	10/15/2024	The {{c1::learning rate}} controls the size of the steps taken in gradient descent.
Published	10/15/2024	Feature scaling using {{c1::mean normalization}} involves adjusting each feature so that its mean is zero.
Published	10/15/2024	Polynomial regression can be used to model {{c1::nonlinear relationships}} between features and the target variable.
Published	10/15/2024	The formula for polynomial regression with degree 2 is: {{c1::\(f_{w,b}(x) = w_1 x + w_2 x^2 + b\)}}.
Published	10/15/2024	When using polynomial regression, it is important to apply {{c1::feature scaling}} to avoid slow convergence in gradient descent.
Published	10/15/2024	In the context of linear regression, a {{c1::residual}} is the difference between the predicted value and the actual value: {{c2::\(e = y - \hat{y}\)}…
Published	10/15/2024	The cost function in ridge regression (L2 regularization) is: {{c1::\(J(w,b) = \frac{1}{2m} \sum (f_{w,b}(x^{(i)}) - y^{(i)})^2 + \lambda \sum w_j^2\)…
Published	10/15/2024	In the context of machine learning, {{c1::overfitting}} occurs when a model performs well on training data but poorly on new, unseen data.
Published	10/15/2024	To avoid overfitting, one can use techniques like {{c1::regularization}}, {{c2::cross-validation}}, and {{c3::early stopping}}.
Published	10/15/2024	The purpose of {{c1::cross-validation}} is to assess the model's performance on unseen data by splitting the dataset into training and validation…
Published	10/15/2024	The {{c1::bias-variance tradeoff}} refers to the balance between underfitting (high bias) and overfitting (high variance) in a model.
Status	Last Update	Fields