How to Calculate MAE, RMSE and R² in scikit-learn
A regression model predicts numbers, so you measure it by how far its numbers are from the real ones. Three scores do most of the work: MAE, RMSE and R². This guide calculates all three with scikit-learn for a real house price model, compares them against a model that always guesses the average, and explains what each number says in plain words.
Syntax
from sklearn.metrics import mean_absolute_error, r2_score, root_mean_squared_error
mae = mean_absolute_error(y_true, y_pred)
rmse = root_mean_squared_error(y_true, y_pred)
r2 = r2_score(y_true, y_pred)
| Part | What it means |
|---|---|
y_true | The real values, such as the test set's target. It always comes first |
y_pred | The model's predictions for the same rows, in the same order |
mean_absolute_error | MAE: the average size of the errors, in the target's own units. "The best value is 0.0" |
root_mean_squared_error | RMSE: like MAE, but large errors count more. Also best at 0.0 |
r2_score | R²: how much of the variation the model explains. "Best possible score is 1.0 and it can be negative" |
Prerequisites
- A uv project with scikit-learn and pandas. See How to Add a Package to a Python Project with uv.
- A trained model and a test set. This guide uses the random forest from How to Train a Random Forest Regression Model with scikit-learn.
Step 1: Score Two Models
A score on its own is hard to judge, so this script scores two models: a random forest, and a baseline that ignores the features and always predicts the average house value of the training rows (DummyRegressor).
from sklearn.datasets import fetch_california_housing
from sklearn.dummy import DummyRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import (
mean_absolute_error,
mean_squared_error,
r2_score,
root_mean_squared_error,
)
from sklearn.model_selection import train_test_split
frame = fetch_california_housing(as_frame=True).frame
X, y = frame.drop(columns="MedHouseVal"), frame["MedHouseVal"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
models = {
"Random forest": RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1),
"Always guess the average": DummyRegressor(strategy="mean"),
}
for name, model in models.items():
predictions = model.fit(X_train, y_train).predict(X_test)
print(name)
print(f" MAE {mean_absolute_error(y_test, predictions):.3f}")
print(f" MSE {mean_squared_error(y_test, predictions):.3f}")
print(f" RMSE {root_mean_squared_error(y_test, predictions):.3f}")
print(f" R2 {r2_score(y_test, predictions):.3f}")
Every metric takes the same two things: the true values (y_test) first, then the predictions.
uv run python regression_metrics.py
Random forest
MAE 0.328
MSE 0.256
RMSE 0.506
R2 0.804
Always guess the average
MAE 0.906
MSE 1.311
RMSE 1.145
R2 -0.000
Step 2: Read Each Score
The house values are in units of $100,000, and so are MAE and RMSE.
MAE, mean absolute error. The average size of a miss. The forest's 0.328 means it's off by about $32,800 per district on average; the baseline is off by about $90,600. Lower is better.
MSE, mean squared error. The average of the squared misses. Squaring makes big misses count much more than small ones, but it also squares the units, so 0.256 is hard to picture. Lower is better.
RMSE, root mean squared error. The square root of MSE: √0.256 ≈ 0.506. It's back in the target's units, about $50,600, and like MSE it punishes big misses. RMSE is always at least as large as MAE; a big gap between them means a few large misses. Lower is better.
R², the coefficient of determination. How much of the variation in house values the model explains, compared with always guessing the average. 1.0 is perfect, 0 is no better than the average, and below 0 is worse than the average. The forest's 0.804 means it explains about 80% of the variation. The baseline's -0.000 is the average-guesser itself: about zero by definition, here a hair below, because the test rows' average differs slightly from the training rows'.
Step 3: Choose the Right Score
| Score | Units | Best for |
|---|---|---|
| MAE | Same as the target | Explaining a typical error to anyone: "off by about $32,800" |
| RMSE | Same as the target | When large misses are much worse than small ones |
| R² | None (0 to 1, can go negative) | Comparing how well models explain the data, across datasets |
Track all three: the MLflow series logs MAE, RMSE and R² for every run and compares them side by side: MLflow Tutorial for Machine Learning in Python — Part 1.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
mean_squared_error(y_true, y_pred, squared=False) from older tutorials | TypeError: got an unexpected keyword argument 'squared' in scikit-learn 1.9.1 | Use root_mean_squared_error: the Fixed post explains why |
| Passing the arguments in the wrong order | R² comes out wrong: 0.748 instead of 0.804 here, while MAE and RMSE don't change | Always true values first: r2_score(y_test, predictions) |
| Scoring on the training rows | Every score looks much better than reality | Score on the test set |
| Reading MSE in dollars | Its units are squared | Read RMSE or MAE instead |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
An ImportError for root_mean_squared_error | scikit-learn older than 1.4, the version that added it | Upgrade scikit-learn |
| R² is negative | The model is worse than always guessing the average | Check the data split, the features and the model |
FAQ
Which score should I report?
MAE for people, because it's in real units and easy to picture. Keep RMSE and R² too: RMSE catches large misses, and R² compares models.
Why is RMSE bigger than MAE?
Squaring weights big misses more, so RMSE is always at least as large as MAE. They're equal only when every miss is the same size.
Is an R² of 0.804 good?
It depends on the problem. Here it's far better than the baseline's 0, and more trees didn't improve it much, which suggests the remaining error comes from information the features don't contain.
Conclusion
mean_absolute_error, root_mean_squared_error and r2_score measured the forest at an average miss of about $32,800, an RMSE of about $50,600 and an R² of 0.804, against $90,600, $114,500 and about 0 for a model that always guesses the average.
Additional Resources
- Regression metrics — scikit-learn's guide to MAE, MSE, RMSE, R² and more.
- root_mean_squared_error — added in scikit-learn 1.4.
- DummyRegressor — the always-guess baseline.
