Fixed: "SHAP or matplotlib package is not installed, so model explainability insights will not be logged" in MLflow
Every evaluation I ran with mlflow.models.evaluate printed WARNING mlflow.models.evaluation.evaluators.shap: SHAP or matplotlib package is not installed, so model explainability insights will not be logged. MLflow 3.18.0's default evaluator explains classifiers and regressors with SHAP, an optional package that mlflow doesn't install. There are two fixes, depending on whether you want the explanations: install shap, or turn explainability off.
What I Was Doing
I was writing How to Evaluate a Classification Model with mlflow.models.evaluate, in a project with mlflow, scikit-learn and pandas: a logistic regression on scikit-learn's breast cancer data, logged and evaluated on the rows it didn't train on:
import mlflow
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
mlflow.set_tracking_uri("http://127.0.0.1:8080")
mlflow.set_experiment("evaluate-classifier-demo")
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
y = y == 0 # True when the tumour is malignant
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
model = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
with mlflow.start_run(run_name="evaluate-logistic"):
info = mlflow.sklearn.log_model(model, name="model", input_example=X_train.head(3))
result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(malignant=y_test),
targets="malignant",
model_type="classifier",
)
for name, value in result.metrics.items():
print(f"{name}: {value:.4f}")
uv run python evaluate_classifier.py
What I Expected
The metrics and plots, without a warning: the project has every package the evaluation guide's example imports.
What I Got
The metrics, after a warning from MLflow's SHAP step:
2026/11/21 18:58:19 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
2026/11/21 18:58:21 WARNING mlflow.models.evaluation.evaluators.shap: SHAP or matplotlib package is not installed, so model explainability insights will not be logged.
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/2a389ef651e14f729b752ce6db3cbcac
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5
score: 0.9580
...
roc_auc: 0.9914
precision_recall_auc: 0.9889

What I Checked
What the default evaluator does. The reference for mlflow.models.evaluate says: "For both the "regressor" and "classifier" model types, the default evaluator generates model summary plots and feature importance plots using SHAP." Its evaluator_config option log_model_explainability is "A boolean value specifying whether or not to log model explainability insights, default value is True."
Which package is missing. The warning comes from mlflow/models/evaluation/evaluators/shap.py, which tries import shap and from matplotlib import pyplot, and logs this warning when either import fails. matplotlib came with mlflow; shap didn't, so uv add mlflow never installed it.
When it runs. For every classifier and regressor evaluation while log_model_explainability is on, the default.
Why It Happens
Explaining a model is on by default in mlflow.models.evaluate, but the package that does it, shap, isn't one of mlflow's requirements. Without it, MLflow skips the explanations and warns, on every evaluation.
The Fix
Pick the fix that matches what you want from the evaluation.
You want the explanations: install shap. I did it in a copy of the project, shap-demo, to keep the original as it was:
uv add shap
Resolved 97 packages in 2.10s
Built shap-demo @ file:///Users/you/source/shap-demo
Prepared 4 packages in 454ms
Uninstalled 1 package in 0.95ms
Installed 4 packages in 4ms
+ shap==0.54.1
~ shap-demo==0.1.0 (from file:///Users/you/source/shap-demo)
+ slicer==0.0.8
+ tqdm==4.70.1

You don't need explanations: turn them off with one line in the evaluate call:

result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(malignant=y_test),
targets="malignant",
model_type="classifier",
evaluator_config={"log_model_explainability": False},
)
The rest of the script is unchanged.
After the Fix
With shap installed, the same script, in shap-demo:
uv run python evaluate_classifier.py
2026/11/21 18:58:36 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
Background dataset has 143 samples but max_samples=100. Subsampling to 100 samples for SHAP value computation. To use all samples, set max_samples=143 when initializing the masker.
2026/11/21 18:58:38 INFO mlflow.models.evaluation.evaluators.shap: Shap explainer PermutationExplainer is used.
/Users/you/source/shap-demo/.venv/lib/python3.15/site-packages/mlflow/models/evaluation/evaluators/shap.py:251: FutureWarning: The NumPy global RNG was seeded by calling `np.random.seed`. In a future version this function will no longer use the global RNG. Pass `rng` explicitly to opt-in to the new behaviour and silence this warning.
shap.summary_plot(shap_values, show=False, color_bar=True)
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/a7239cf5f7ac4e349e903bf9ac2d5dd1
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5

The warning is gone, and the run has three more plots: shap_beeswarm_plot.png, shap_feature_importance_plot.png and shap_summary_plot.png. Two new lines come with them:
Background dataset has 143 samples but max_samples=100: shap's note that it explains the model from 100 of the 143 test rows.FutureWarning: The NumPy global RNG was seeded...: raised by shap's ownsummary_plotfunction, which MLflow calls atshap.pyline 251 without therngargument shap asks for. The global generator was seeded by shap itself: itsPermutationExplainer, which explained this model, callsnp.random.seed, as shap issue #4607 records. It doesn't change the plots, and neither call is in your code.
With explainability off, back in the original project:
uv run python evaluate_no_shap.py
2026/11/21 18:58:50 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/3ce4adf28d9444f8b5c71ac746f09338
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5
score: 0.9580
...
roc_auc: 0.9914
precision_recall_auc: 0.9889
The output goes straight from Testing metrics on first row... to the run's link, with the same metrics and no warning.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
Installing shap for a model with columns of words | TypeError: Cannot interpret '<StringDtype(na_value=nan)>' as a data type before SHAP runs | log_model_explainability: False: Fixed: Cannot interpret StringDtype |
| Leaving the warning in every evaluation | Real warnings get lost among them | Install shap or turn explanations off |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
SHAP or matplotlib package is not installed | No shap in the project | uv add shap, or log_model_explainability: False |
FutureWarning: The NumPy global RNG was seeded by calling np.random.seed after installing shap | shap's summary_plot, called by MLflow, after shap's PermutationExplainer seeded NumPy's global generator (shap issue #4607) | Nothing to change in your code |
FAQ
What do the SHAP plots show?
Which columns move the model's predictions, and in which direction: How to Log SHAP Explanations with mlflow.models.evaluate walks through them.
Does installing shap slow the evaluation down?
It adds the explanation step: in this run, three seconds between the first-row test and the explainer's line.
Conclusion
The SHAP warning means explanations were skipped because the shap package isn't installed. uv add shap brought the three SHAP plots, with a harmless FutureWarning from shap's own code; evaluator_config={"log_model_explainability": False} removed the warning for evaluations that don't need explanations.
Additional Resources
- mlflow.models.evaluate — the default evaluator's SHAP plots and
log_model_explainability. - SHAP documentation — what SHAP values are.
