Fixed: "TypeError: unsupported format string passed to MetricValue.__format__" in an MLflow Custom Artifact
I added a custom plot to mlflow.models.evaluate and put the model's R² in its title, read from the builtin_metrics argument. The evaluation crashed with TypeError: unsupported format string passed to MetricValue.__format__. MLflow 3.18.0's reference types builtin_metrics as a dictionary of numbers, but each value is a MetricValue object, with the number inside. Reading it from aggregate_results fixed the plot.
What I Was Doing
I was writing How to Add a Custom Plot to an MLflow Evaluation: a linear regression on scikit-learn's diabetes data, evaluated with a custom artifact function that plots each prediction against the true value. The reference for mlflow.models.evaluate gives the function's signature as custom_artifact(eval_df, builtin_metrics: Dict[str, float], artifacts_dir), so I formatted the R² from builtin_metrics as a number:
from pathlib import Path
import matplotlib.pyplot as plt
import mlflow
from sklearn.datasets import load_diabetes
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
mlflow.set_tracking_uri("http://127.0.0.1:8080")
mlflow.set_experiment("custom-plot-demo")
def predicted_vs_actual(eval_df, builtin_metrics, artifacts_dir):
"""Plot each prediction against the true value, and save the plot as a file."""
fig, ax = plt.subplots(figsize=(6, 4.5))
ax.scatter(eval_df["target"], eval_df["prediction"], alpha=0.6)
ax.set_xlabel("True progression")
ax.set_ylabel("Predicted progression")
ax.set_title(f"R² = {builtin_metrics['r2_score']:.3f}")
path = Path(artifacts_dir) / "predicted_vs_actual.png"
fig.savefig(path)
plt.close(fig)
return {"predicted_vs_actual": str(path)}
X, y = load_diabetes(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
model = LinearRegression().fit(X_train, y_train)
with mlflow.start_run(run_name="custom-plot"):
info = mlflow.sklearn.log_model(model, name="model", input_example=X_train.head(3))
result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(progression=y_test),
targets="progression",
model_type="regressor",
custom_artifacts=[predicted_vs_actual],
)
print(result.artifacts["predicted_vs_actual"].uri)
uv run python custom_plot.py
What I Expected
A plot titled with the R², logged to the run, and its address printed.
What I Got
A traceback ending in my function's set_title line:
2026/11/21 18:02:59 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
🏃 View run custom-plot at: http://127.0.0.1:8080/#/experiments/9/runs/fec9cdc8b7034b66b47c42b8a73c6f82
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/9
Traceback (most recent call last):
File "/Users/you/source/mlflow-demo/custom_plot.py", line 32, in <module>
...
File "/Users/you/source/mlflow-demo/custom_plot.py", line 19, in predicted_vs_actual
ax.set_title(f"R² = {builtin_metrics['r2_score']:.3f}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unsupported format string passed to MetricValue.__format__

The run ended FAILED, with no metrics and no files: MLflow builds the custom plots before it logs anything.
What I Checked
What the reference promises. Under custom_artifacts, the mlflow.models.evaluate reference types the argument builtin_metrics: Dict[str, float] and describes it as "A dictionary containing the metrics calculated by the default evaluator. The keys are the names of the metrics and the values are the scalar values of the metrics."
What the error says. unsupported format string passed to MetricValue.__format__: the value isn't a number but a MetricValue, which can't be formatted with :.3f.
What MLflow passes. In mlflow/models/evaluation/default_evaluator.py, the function is called with a copy of the evaluator's metrics_values, and the built-in metrics go into that dictionary through _get_aggregate_metrics_values, which wraps each one: MetricValue(aggregate_results={name: value}). The reference for mlflow.metrics.MetricValue describes that field: "aggregate_results – A dictionary mapping the name of the aggregation to its value".
Why It Happens
MLflow 3.18.0 keeps every metric as a MetricValue, and hands the custom artifact function those objects, not the plain numbers the reference describes. For a built-in metric, the number sits in aggregate_results, under the metric's own name. I reported the reference's wording to MLflow as issue #28014.
The Fix
Read the number from aggregate_results:

def predicted_vs_actual(eval_df, builtin_metrics, artifacts_dir):
"""Plot each prediction against the true value, and save the plot as a file."""
fig, ax = plt.subplots(figsize=(6, 4.5))
ax.scatter(eval_df["target"], eval_df["prediction"], alpha=0.6)
ax.set_xlabel("True progression")
ax.set_ylabel("Predicted progression")
# Each built-in metric arrives as a MetricValue; its number is in aggregate_results.
r2 = builtin_metrics["r2_score"].aggregate_results["r2_score"]
ax.set_title(f"R² = {r2:.3f}")
path = Path(artifacts_dir) / "predicted_vs_actual.png"
fig.savefig(path)
plt.close(fig)
return {"predicted_vs_actual": str(path)}
After the Fix
uv run python custom_plot.py
2026/11/21 18:57:47 INFO mlflow.models.evaluation.default_evaluator: Testing metrics on first row...
2026/11/21 18:57:48 WARNING mlflow.models.evaluation.evaluators.shap: SHAP or matplotlib package is not installed, so model explainability insights will not be logged.
🏃 View run custom-plot at: http://127.0.0.1:8080/#/experiments/9/runs/8de8367e278a4692989dbfcc45572194
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/9
mlflow-artifacts:/9/8de8367e278a4692989dbfcc45572194/artifacts/predicted_vs_actual.png

The plot was logged, titled R² = 0.359. The SHAP line is unrelated: Fixed: SHAP or matplotlib package is not installed covers it.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
f"{builtin_metrics['r2_score']:.3f}" | TypeError: unsupported format string passed to MetricValue.__format__ | builtin_metrics["r2_score"].aggregate_results["r2_score"] |
float(builtin_metrics["r2_score"]) | TypeError: float() argument must be a string or a real number, not 'MetricValue' | Read aggregate_results |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
TypeError: unsupported format string passed to MetricValue.__format__ in a custom artifact | Formatting a MetricValue as a number | Read aggregate_results[name] |
TypeError: float() argument must be a string or a real number, not 'MetricValue' | Converting a MetricValue with float() | The same |
FAQ
Do custom metrics get MetricValue objects too?
A custom metric's metrics argument gets the same dictionary of MetricValue objects; the make_metric reference says so for that argument, "To access the MetricValue for the metrics calculated by the system, make sure to specify the type hint for this parameter as Dict[str, MetricValue]."
Why didn't the run log anything?
MLflow builds the custom artifacts before it logs the metrics, so the crash came first.
Conclusion
The custom plot crashed because builtin_metrics holds MetricValue objects, not the floats the reference describes. Reading the number from builtin_metrics["r2_score"].aggregate_results["r2_score"] let the plot carry its R², and the evaluation logged it.
Additional Resources
- mlflow.models.evaluate — the
custom_artifactsparameter. - mlflow.metrics.MetricValue —
scores,justificationsandaggregate_results.
