Fixed: scikit-learn Random Forest Predictions Change Slightly Between Calls with n_jobs=-1
A check that two random forests predict the same values printed False, although both forests had exactly the same trees. Even one forest, asked twice for the same predictions, gave values that differed in the 15th decimal place. With n_jobs=-1, scikit-learn adds up the trees' predictions in whatever order its threads finish, and decimal rounding makes the order show. Compare predictions with np.allclose, or predict with n_jobs=1 when you need identical numbers.
What I Was Doing
While writing How to Use warm_start to Add Trees to a Random Forest in scikit-learn, I grew a RandomForestRegressor to 100 trees in four stages with warm_start=True, on the California Housing data, with scikit-learn 1.9.1 and Python 3.15.2. To prove the staged forest matches one grown in a single call, the script ended with this check:
all_at_once = RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1)
all_at_once.fit(X_train, y_train)
same = np.array_equal(forest.predict(X_val), all_at_once.predict(X_val))
print("Same predictions as 100 trees grown at once:", same)
uv run python warm_start_forest.py
What I Expected
True. Both forests use random_state=42, and growing in stages is meant to give the same trees.
What I Got
25 trees: validation RMSE 0.5127
50 trees: validation RMSE 0.5070
75 trees: validation RMSE 0.5083
100 trees: validation RMSE 0.5063
Same predictions as 100 trees grown at once: False

What I Checked
Are the trees the same? Yes. I compared the two forests tree by tree: every tree had the same random seed, and the same split points and leaf values. scikit-learn's source code, in sklearn/ensemble/_forest.py, gives warm-started trees "the random state we would have got if we hadn't used a warm_start."
How far apart are the predictions? The largest difference was 1.7763568394002505e-15, a change in the 15th decimal place.
Does one forest even agree with itself? This script asks one forest for the same predictions twice, first with n_jobs=-1, then with n_jobs=1:
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
X, y = fetch_california_housing(return_X_y=True, as_frame=True)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
forest = RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1)
forest.fit(X_train, y_train)
first = forest.predict(X_val)
second = forest.predict(X_val)
print("n_jobs=-1, same forest, same rows:")
print(" np.array_equal:", np.array_equal(first, second))
print(" np.allclose: ", np.allclose(first, second))
print(" largest difference:", np.abs(first - second).max())
forest.set_params(n_jobs=1)
print("n_jobs=1, same forest, same rows:")
print(" np.array_equal:", np.array_equal(forest.predict(X_val), forest.predict(X_val)))
uv run python check_forest.py
n_jobs=-1, same forest, same rows:
np.array_equal: False
np.allclose: True
largest difference: 1.7763568394002505e-15
n_jobs=1, same forest, same rows:
np.array_equal: True

So it isn't about warm starts at all: one forest, predicting with every processor core, doesn't give the same last digits twice. With one core it does.
How the forest predicts. In scikit-learn 1.9.1, RandomForestRegressor.predict starts all trees in parallel and adds each one's prediction into a single running total as soon as it's ready, then divides by the number of trees:
# Parallel loop
lock = threading.Lock()
Parallel(n_jobs=n_jobs, verbose=self.verbose, require="sharedmem")(
delayed(_accumulate_prediction)(e.predict, X, [y_hat], lock)
for e in self.estimators_
)
y_hat /= len(self.estimators_)
The lock stops two threads writing at the same moment, but not the order: whichever tree finishes first is added first.
Why It Happens
Computers store decimal numbers in binary with limited precision, so, as Python's documentation puts it, "every float operation can suffer a new rounding error." Adding the same 100 numbers in a different order rounds at different moments, and the total can come out different in its last digit. With n_jobs=-1, the order changes from call to call; with n_jobs=1, one thread adds the trees in the same order every time.
The difference, about 0.0000000000000018, is far below anything that matters for a house price. It only matters to a check that demands every bit to be equal.
The Fix
Compare predictions with a tolerance. NumPy's np.allclose "Returns True if two arrays are element-wise equal within a tolerance":
same = np.allclose(forest.predict(X_val), all_at_once.predict(X_val))
print("Same predictions as 100 trees grown at once:", same)
Its defaults, rtol=1e-05 and atol=1e-08, are millions of times larger than the rounding differences here, and still small enough to catch a forest that really predicts something else.
When you need bit-for-bit identical numbers, for example to compare saved predictions in a test, predict with one thread: forest.set_params(n_jobs=1) before calling predict.
After the Fix
uv run python warm_start_forest.py
25 trees: validation RMSE 0.5127
50 trees: validation RMSE 0.5070
75 trees: validation RMSE 0.5083
100 trees: validation RMSE 0.5063
Same predictions as 100 trees grown at once: True
The staged forest and the one grown at once agree, as their identical trees said they should.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
Comparing model outputs with np.array_equal or == | Differences in the last digit fail the check | Use np.allclose |
Blaming random_state or warm_start | Both were working: the trees were identical | Check the trees first, then the way predictions are added up |
Setting n_jobs=1 everywhere to "fix" it | Training and prediction use one core and get slower | Keep n_jobs=-1 and compare with a tolerance; use one thread only where identical digits are required |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Differences around 1e-15 or smaller | The order of adding up the trees' predictions | Expected with n_jobs=-1; compare with np.allclose |
np.allclose is False too | The forests really differ: different data, settings or random_state | Check that both forests were fitted on the same rows with the same options |
FAQ
Does this make random forests unreliable?
No. Each prediction is the same to about 15 significant digits. Only checks that demand identical bits notice.
Does it happen with n_jobs=1?
Not in this test: with one thread, two calls returned exactly equal predictions.
Do other scikit-learn models do this?
Any model that adds up results from parallel threads can show the same last-digit differences. Comparing with a tolerance works for all of them.
Conclusion
np.array_equal said a warm-started forest and a forest grown in one call predicted different values, but the trees were identical, and one forest didn't even agree with itself from call to call. With n_jobs=-1, scikit-learn adds the trees' predictions in the order its threads finish, and floating-point rounding changed the 15th decimal place. np.allclose confirmed the forests match, and n_jobs=1 gives identical numbers when a check needs them.
Additional Resources
- Floating-Point Arithmetic: Issues and Limitations — Python's explanation of rounding in decimal numbers.
- numpy.allclose — NumPy's reference, with
rtolandatol. - RandomForestRegressor — the reference, including
n_jobs.
