What is the bias-variance tradeoff in machine learning

Short answer

High bias causes underfitting and poor training performance, while high variance causes overfitting and poor generalization. The tradeoff is balancing model complexity to minimize total error. For evaluating where your model sits, see how to evaluate machine learning model performance. For improving data quality, see what is feature engineering and why is it important.

Steps

1. Check training error: high error indicates high bias
2. Check test error: large gap between train and test indicates high variance
3. Reduce bias by increasing model complexity or adding features
4. Reduce variance with regularization, more data, or simpler models
5. Use validation curves to find the sweet spot

Tips

• Ensemble methods can reduce variance without adding much bias
• Cross-validation helps estimate true generalization error
• For data preparation strategies, see how to preprocess data for machine learning models