Scenario
You find that coating thickness is related to coating viscosity, coating speed, and temperature. Your supervisor asks: "If I adjust the speed from 10 to 12, how will the thickness change?" You've only done a simple correlation analysis, can't consider three factors simultaneously, and can't answer.
This is the problem multiple regression aims to solve.
Simple Regression vs. Multiple Regression
Simple Linear Regression: Y = a + b × X
Considers only one input variable.
Multiple Linear Regression: Y = a + b₁X₁ + b₂X₂ + b₃X₃ + ...
Considers multiple input variables simultaneously, with each factor having its own coefficient (degree of influence).
Interpreting Key Metrics
R² (Coefficient of Determination)
- Range 0-1, represents how much variation the model explains.
- R² = 0.85: The model explains 85% of the data variation.
- In general manufacturing, R² > 0.7 has practical value.
Adjusted R²
- Adding more factors will always increase R² (even if the factor is useless).
- Adjusted R² penalizes useless factors, making it more reliable than R².
P-value (for each factor)
- P < 0.05: This factor has a significant impact on Y.
- P > 0.05: The influence of this factor might just be random; consider removing it from the model.
VIF (Variance Inflation Factor)
- Detects multicollinearity (two input factors are highly correlated).
- VIF > 10: Indicates a serious multicollinearity problem, and model coefficients are unreliable.
Practical Steps
Step 1: Check scatter plots of each factor and Y
- Look for linear relationships.
- Identify obvious outliers.
Step 2: Build an initial model (include all possible factors)
Step 3: Filter using P-values
- Remove factors with P > 0.05.
- Rebuild the model, repeat until all factors are significant.
Step 4: Verify model assumptions
- Residual plot: Should be randomly distributed, with no patterns.
- Normal probability plot: Residuals should fall on a straight line.
- Residuals vs. predicted values: Should show no patterns.
Step 5: Use the model for prediction
- Substitute new X values to get the predicted Y value and confidence interval.
Common Pitfalls
| Pitfall | Explanation | Solution |
|---|---|---|
| Extrapolation | Predicting outside the original data range | Predict only within the data range |
| Collinearity | X1 and X2 are highly correlated | Keep only one or use PCA |
| Overfitting | Too many factors, poor prediction for new data | Control with Adjusted R² |
| Ignoring interaction | A and B jointly affect Y | Add A×B interaction term |
Golden Quote
"A regression equation is not the truth; it is the best approximation within your data range—outside this range, please be humble."