Skip to main content

Fixed: scikit-learn Random Forest Predictions Change Slightly Between Calls with n_jobs=-1

· 7 min read
Jagdish Kumawat
Founder @ Dewiride

A check that two random forests predict the same values printed False, although both forests had exactly the same trees. Even one forest, asked twice for the same predictions, gave values that differed in the 15th decimal place. With n_jobs=-1, scikit-learn adds up the trees' predictions in whatever order its threads finish, and decimal rounding makes the order show. Compare predictions with np.allclose, or predict with n_jobs=1 when you need identical numbers.

What I Was Doing​

While writing How to Use warm_start to Add Trees to a Random Forest in scikit-learn, I grew a RandomForestRegressor to 100 trees in four stages with warm_start=True, on the California Housing data, with scikit-learn 1.9.1 and Python 3.15.2. To prove the staged forest matches one grown in a single call, the script ended with this check:

warm_start_forest.py (the last lines)
all_at_once = RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1)
all_at_once.fit(X_train, y_train)
same = np.array_equal(forest.predict(X_val), all_at_once.predict(X_val))
print("Same predictions as 100 trees grown at once:", same)
Terminal
uv run python warm_start_forest.py

What I Expected​

True. Both forests use random_state=42, and growing in stages is meant to give the same trees.

What I Got​

Output
25 trees: validation RMSE 0.5127
50 trees: validation RMSE 0.5070
75 trees: validation RMSE 0.5083
100 trees: validation RMSE 0.5063
Same predictions as 100 trees grown at once: False

A terminal running uv run python warm_start_forest.py: the validation RMSE at 25, 50, 75 and 100 trees, then the line Same predictions as 100 trees grown at once: False.

What I Checked​

Are the trees the same? Yes. I compared the two forests tree by tree: every tree had the same random seed, and the same split points and leaf values. scikit-learn's source code, in sklearn/ensemble/_forest.py, gives warm-started trees "the random state we would have got if we hadn't used a warm_start."

How far apart are the predictions? The largest difference was 1.7763568394002505e-15, a change in the 15th decimal place.

Does one forest even agree with itself? This script asks one forest for the same predictions twice, first with n_jobs=-1, then with n_jobs=1:

check_forest.py
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

X, y = fetch_california_housing(return_X_y=True, as_frame=True)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

forest = RandomForestRegressor(n_estimators=100, random_state=42, n_jobs=-1)
forest.fit(X_train, y_train)

first = forest.predict(X_val)
second = forest.predict(X_val)
print("n_jobs=-1, same forest, same rows:")
print(" np.array_equal:", np.array_equal(first, second))
print(" np.allclose: ", np.allclose(first, second))
print(" largest difference:", np.abs(first - second).max())

forest.set_params(n_jobs=1)
print("n_jobs=1, same forest, same rows:")
print(" np.array_equal:", np.array_equal(forest.predict(X_val), forest.predict(X_val)))
Terminal
uv run python check_forest.py
Output
n_jobs=-1, same forest, same rows:
np.array_equal: False
np.allclose: True
largest difference: 1.7763568394002505e-15
n_jobs=1, same forest, same rows:
np.array_equal: True

A terminal running uv run python check_forest.py: with n_jobs=-1 the same forest's two predictions are not exactly equal but are allclose, with a largest difference of 1.7763568394002505e-15, and with n_jobs=1 they are exactly equal.

So it isn't about warm starts at all: one forest, predicting with every processor core, doesn't give the same last digits twice. With one core it does.

How the forest predicts. In scikit-learn 1.9.1, RandomForestRegressor.predict starts all trees in parallel and adds each one's prediction into a single running total as soon as it's ready, then divides by the number of trees:

sklearn/ensemble/_forest.py (scikit-learn 1.9.1, RandomForestRegressor.predict)
# Parallel loop
lock = threading.Lock()
Parallel(n_jobs=n_jobs, verbose=self.verbose, require="sharedmem")(
delayed(_accumulate_prediction)(e.predict, X, [y_hat], lock)
for e in self.estimators_
)

y_hat /= len(self.estimators_)

The lock stops two threads writing at the same moment, but not the order: whichever tree finishes first is added first.

Why It Happens​

Computers store decimal numbers in binary with limited precision, so, as Python's documentation puts it, "every float operation can suffer a new rounding error." Adding the same 100 numbers in a different order rounds at different moments, and the total can come out different in its last digit. With n_jobs=-1, the order changes from call to call; with n_jobs=1, one thread adds the trees in the same order every time.

The difference, about 0.0000000000000018, is far below anything that matters for a house price. It only matters to a check that demands every bit to be equal.

The Fix​

Compare predictions with a tolerance. NumPy's np.allclose "Returns True if two arrays are element-wise equal within a tolerance":

warm_start_forest.py (the last lines, fixed)
same = np.allclose(forest.predict(X_val), all_at_once.predict(X_val))
print("Same predictions as 100 trees grown at once:", same)

Its defaults, rtol=1e-05 and atol=1e-08, are millions of times larger than the rounding differences here, and still small enough to catch a forest that really predicts something else.

When you need bit-for-bit identical numbers, for example to compare saved predictions in a test, predict with one thread: forest.set_params(n_jobs=1) before calling predict.

After the Fix​

Terminal
uv run python warm_start_forest.py
Output
25 trees: validation RMSE 0.5127
50 trees: validation RMSE 0.5070
75 trees: validation RMSE 0.5083
100 trees: validation RMSE 0.5063
Same predictions as 100 trees grown at once: True

The staged forest and the one grown at once agree, as their identical trees said they should.

Common Mistakes​

MistakeWhat happensDo this instead
Comparing model outputs with np.array_equal or ==Differences in the last digit fail the checkUse np.allclose
Blaming random_state or warm_startBoth were working: the trees were identicalCheck the trees first, then the way predictions are added up
Setting n_jobs=1 everywhere to "fix" itTraining and prediction use one core and get slowerKeep n_jobs=-1 and compare with a tolerance; use one thread only where identical digits are required

Troubleshooting​

SymptomLikely causeFix
Differences around 1e-15 or smallerThe order of adding up the trees' predictionsExpected with n_jobs=-1; compare with np.allclose
np.allclose is False tooThe forests really differ: different data, settings or random_stateCheck that both forests were fitted on the same rows with the same options

FAQ​

Does this make random forests unreliable?​

No. Each prediction is the same to about 15 significant digits. Only checks that demand identical bits notice.

Does it happen with n_jobs=1?​

Not in this test: with one thread, two calls returned exactly equal predictions.

Do other scikit-learn models do this?​

Any model that adds up results from parallel threads can show the same last-digit differences. Comparing with a tolerance works for all of them.

Conclusion​

np.array_equal said a warm-started forest and a forest grown in one call predicted different values, but the trees were identical, and one forest didn't even agree with itself from call to call. With n_jobs=-1, scikit-learn adds the trees' predictions in the order its threads finish, and floating-point rounding changed the 15th decimal place. np.allclose confirmed the forests match, and n_jobs=1 gives identical numbers when a check needs them.

Additional Resources​