Skip to main content

Fixed: "SHAP or matplotlib package is not installed, so model explainability insights will not be logged" in MLflow

· 8 min read
Jagdish Kumawat
Founder @ Dewiride

Every evaluation I ran with mlflow.models.evaluate printed WARNING mlflow.models.evaluation.evaluators.shap: SHAP or matplotlib package is not installed, so model explainability insights will not be logged. MLflow 3.18.0's default evaluator explains classifiers and regressors with SHAP, an optional package that mlflow doesn't install. There are two fixes, depending on whether you want the explanations: install shap, or turn explainability off.

What I Was Doing​

I was writing How to Evaluate a Classification Model with mlflow.models.evaluate, in a project with mlflow, scikit-learn and pandas: a logistic regression on scikit-learn's breast cancer data, logged and evaluated on the rows it didn't train on:

evaluate_classifier.py
import mlflow
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

mlflow.set_tracking_uri("http://127.0.0.1:8080")
mlflow.set_experiment("evaluate-classifier-demo")

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
y = y == 0 # True when the tumour is malignant
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
model = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)

with mlflow.start_run(run_name="evaluate-logistic"):
info = mlflow.sklearn.log_model(model, name="model", input_example=X_train.head(3))
result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(malignant=y_test),
targets="malignant",
model_type="classifier",
)

for name, value in result.metrics.items():
print(f"{name}: {value:.4f}")
Terminal
uv run python evaluate_classifier.py

What I Expected​

The metrics and plots, without a warning: the project has every package the evaluation guide's example imports.

What I Got​

The metrics, after a warning from MLflow's SHAP step:

Output (excerpt)
2026/11/21 18:58:19 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
2026/11/21 18:58:21 WARNING mlflow.models.evaluation.evaluators.shap: SHAP or matplotlib package is not installed, so model explainability insights will not be logged.
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/2a389ef651e14f729b752ce6db3cbcac
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5
score: 0.9580
...
roc_auc: 0.9914
precision_recall_auc: 0.9889

The output in a terminal: Testing metrics on first row, WARNING SHAP or matplotlib package is not installed, so model explainability insights will not be logged, the run's link, and the metrics from score 0.9580 to precision_recall_auc 0.9889.

What I Checked​

What the default evaluator does. The reference for mlflow.models.evaluate says: "For both the "regressor" and "classifier" model types, the default evaluator generates model summary plots and feature importance plots using SHAP." Its evaluator_config option log_model_explainability is "A boolean value specifying whether or not to log model explainability insights, default value is True."

Which package is missing. The warning comes from mlflow/models/evaluation/evaluators/shap.py, which tries import shap and from matplotlib import pyplot, and logs this warning when either import fails. matplotlib came with mlflow; shap didn't, so uv add mlflow never installed it.

When it runs. For every classifier and regressor evaluation while log_model_explainability is on, the default.

Why It Happens​

Explaining a model is on by default in mlflow.models.evaluate, but the package that does it, shap, isn't one of mlflow's requirements. Without it, MLflow skips the explanations and warns, on every evaluation.

The Fix​

Pick the fix that matches what you want from the evaluation.

You want the explanations: install shap. I did it in a copy of the project, shap-demo, to keep the original as it was:

Terminal
uv add shap
Output
Resolved 97 packages in 2.10s
Built shap-demo @ file:///Users/you/source/shap-demo
Prepared 4 packages in 454ms
Uninstalled 1 package in 0.95ms
Installed 4 packages in 4ms
+ shap==0.54.1
~ shap-demo==0.1.0 (from file:///Users/you/source/shap-demo)
+ slicer==0.0.8
+ tqdm==4.70.1

The output of uv add shap in a terminal: 97 packages resolved and shap 0.54.1, slicer 0.0.8 and tqdm 4.70.1 installed.

You don't need explanations: turn them off with one line in the evaluate call:

The evaluate call in a macOS-style code window with evaluator_config set to log_model_explainability False after model_type classifier.

evaluate_no_shap.py (the evaluate call)
result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(malignant=y_test),
targets="malignant",
model_type="classifier",
evaluator_config={"log_model_explainability": False},
)

The rest of the script is unchanged.

After the Fix​

With shap installed, the same script, in shap-demo:

Terminal
uv run python evaluate_classifier.py
Output (excerpt)
2026/11/21 18:58:36 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
Background dataset has 143 samples but max_samples=100. Subsampling to 100 samples for SHAP value computation. To use all samples, set max_samples=143 when initializing the masker.
2026/11/21 18:58:38 INFO mlflow.models.evaluation.evaluators.shap: Shap explainer PermutationExplainer is used.
/Users/you/source/shap-demo/.venv/lib/python3.15/site-packages/mlflow/models/evaluation/evaluators/shap.py:251: FutureWarning: The NumPy global RNG was seeded by calling `np.random.seed`. In a future version this function will no longer use the global RNG. Pass `rng` explicitly to opt-in to the new behaviour and silence this warning.
shap.summary_plot(shap_values, show=False, color_bar=True)
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/a7239cf5f7ac4e349e903bf9ac2d5dd1
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5

The output with shap installed in a terminal: Testing metrics on first row, shap subsampling 100 of 143 samples, Shap explainer PermutationExplainer is used, a FutureWarning from shap's summary plot, and the run's link.

The warning is gone, and the run has three more plots: shap_beeswarm_plot.png, shap_feature_importance_plot.png and shap_summary_plot.png. Two new lines come with them:

  • Background dataset has 143 samples but max_samples=100: shap's note that it explains the model from 100 of the 143 test rows.
  • FutureWarning: The NumPy global RNG was seeded...: raised by shap's own summary_plot function, which MLflow calls at shap.py line 251 without the rng argument shap asks for. The global generator was seeded by shap itself: its PermutationExplainer, which explained this model, calls np.random.seed, as shap issue #4607 records. It doesn't change the plots, and neither call is in your code.

With explainability off, back in the original project:

Terminal
uv run python evaluate_no_shap.py
Output (excerpt)
2026/11/21 18:58:50 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
🏃 View run evaluate-logistic at: http://127.0.0.1:8080/#/experiments/5/runs/3ce4adf28d9444f8b5c71ac746f09338
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/5
score: 0.9580
...
roc_auc: 0.9914
precision_recall_auc: 0.9889

The output goes straight from Testing metrics on first row... to the run's link, with the same metrics and no warning.

Common Mistakes​

MistakeWhat happensDo this instead
Installing shap for a model with columns of wordsTypeError: Cannot interpret '<StringDtype(na_value=nan)>' as a data type before SHAP runslog_model_explainability: False: Fixed: Cannot interpret StringDtype
Leaving the warning in every evaluationReal warnings get lost among themInstall shap or turn explanations off

Troubleshooting​

SymptomLikely causeFix
SHAP or matplotlib package is not installedNo shap in the projectuv add shap, or log_model_explainability: False
FutureWarning: The NumPy global RNG was seeded by calling np.random.seed after installing shapshap's summary_plot, called by MLflow, after shap's PermutationExplainer seeded NumPy's global generator (shap issue #4607)Nothing to change in your code

FAQ​

What do the SHAP plots show?​

Which columns move the model's predictions, and in which direction: How to Log SHAP Explanations with mlflow.models.evaluate walks through them.

Does installing shap slow the evaluation down?​

It adds the explanation step: in this run, three seconds between the first-row test and the explainer's line.

Conclusion​

The SHAP warning means explanations were skipped because the shap package isn't installed. uv add shap brought the three SHAP plots, with a harmless FutureWarning from shap's own code; evaluator_config={"log_model_explainability": False} removed the warning for evaluations that don't need explanations.

Additional Resources​