How to Train a Random Forest Regression Model with scikit-learn
A random forest is one of the most dependable models for tables of numbers. It trains many decision trees on random slices of the data and averages their answers, so one odd tree can't throw it off. This guide trains scikit-learn's RandomForestRegressor to predict house values, explains its main settings, and shows which features it relied on, with real output.
Syntax
from sklearn.ensemble import RandomForestRegressor
model = RandomForestRegressor(n_estimators=100, max_depth=None, random_state=None, n_jobs=None)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
| Part | What it means | Default |
|---|---|---|
n_estimators | How many trees the forest has | 100 |
max_depth | How deep each tree may grow. None lets it grow until every leaf is pure or too small to split | None |
random_state | A number that makes training repeatable | None |
n_jobs | How many processor cores to use. -1 means all of them | None, one core |
model.fit(X, y) | Trains the forest on the training rows | |
model.predict(X) | Predicts a value for each row |
Prerequisites
- A uv project with scikit-learn and pandas. See How to Add a Package to a Python Project with uv.
- Training and test sets. This guide uses the split from How to Split Data into Training and Test Sets with scikit-learn.
Step 1: Load and Split the Data
import time
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
frame = fetch_california_housing(as_frame=True).frame
X, y = frame.drop(columns="MedHouseVal"), frame["MedHouseVal"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
Step 2: Create and Train the Forest
Add:
model = RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1)
start = time.perf_counter()
model.fit(X_train, y_train)
print(f"Trained {len(model.estimators_)} trees in {time.perf_counter() - start:.1f} seconds")
| Setting | What it does | Default |
|---|---|---|
n_estimators | How many trees to grow. More trees give steadier answers but take longer | 100 |
max_depth | How deep each tree may grow. None means no limit | None |
random_state | Fixes the randomness, so training twice gives the same forest | None (different every time) |
n_jobs | How many processor cores to use. -1 means all of them | None (one core) |
model.fit(X_train, y_train) trains the forest on the training rows only. model.estimators_ holds the trained trees.
Step 3: Make Predictions
Add:
print("First five test houses:")
for predicted, actual in zip(model.predict(X_test[:5]), y_test[:5], strict=True):
print(f" predicted {predicted:.2f} actual {actual:.2f}")
model.predict() takes rows of features and returns one prediction per row.
Step 4: See Which Features Mattered
Add:
print("Which features mattered most:")
for importance, name in sorted(zip(model.feature_importances_, X.columns, strict=True), reverse=True):
print(f" {name:<10} {importance:.3f}")
Run the script:
uv run python random_forest.py
Trained 100 trees in 1.1 seconds
First five test houses:
predicted 0.51 actual 0.48
predicted 0.73 actual 0.46
predicted 4.94 actual 5.00
predicted 2.51 actual 2.19
predicted 2.27 actual 2.78
Which features mattered most:
MedInc 0.525
AveOccup 0.138
Latitude 0.089
Longitude 0.089
HouseAge 0.055
AveRooms 0.044
Population 0.031
AveBedrms 0.030
Step 5: Read the Result
- Training took 1.1 seconds on the Mac used for this guide, with all cores. Your time will differ.
- The predictions are close but not exact. The third house is predicted at 4.94 against 5.00 ($494,000 against $500,000); the second is off by 0.27. One average score for all 4,128 test houses comes next: How to Calculate MAE, RMSE and R² in scikit-learn.
- Median income dominates.
MedInccarries 0.525 of the importance, more than half. Location (Latitude,Longitude) and household size (AveOccup) come next.
The importances add up to 1. They measure how much each feature helped the trees split the data during training. scikit-learn's documentation warns that these impurity-based importances can be biased toward features with many distinct values, so treat them as a guide, not a proof.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
No random_state | Every training run gives a slightly different forest, so you can't compare runs fairly | Set random_state and record it |
| Training on all rows, then testing on some of them | The score looks better than it really is | Split first, train on X_train only |
Raising n_estimators a lot | Training gets slower for little gain | Compare a few values. scikit-learn's guide notes that "results will stop getting significantly better beyond a critical number of trees" |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Training takes long | n_jobs isn't set, so one core does all the work: 7.1 seconds here, against 1.2 seconds with n_jobs=-1 | n_jobs=-1 |
ValueError: The feature names should match those that were passed during fit. | The prediction rows have different columns, or the same columns in another order | Predict with the same columns, in the same order |
FAQ
Why a random forest for house prices?
It handles features of very different scales without any preparation and captures non-linear patterns, which makes it a strong first model for tables of numbers.
How do I choose n_estimators and max_depth?
Try several values and compare the test scores. Tracking each try with MLflow makes that easy: MLflow Tutorial for Machine Learning in Python — Part 1 compares four forests.
Does a random forest need scaled features?
No. Trees split on thresholds, so the scale of a feature doesn't matter.
Conclusion
RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1) trained 100 trees on 16,512 rows in about a second, predicted house values close to the real ones, and showed that median income drives more than half of its decisions.
Additional Resources
- RandomForestRegressor — every setting and its default.
- Ensembles: random forests — how random forests work.
- Permutation importance vs impurity-based importance — scikit-learn's example of the importance bias.
