Fixed: "AttributeError: module 'mlflow.exceptions' has no attribute 'ModelValidationFailedException'"
I checked a model against minimum scores with mlflow.validate_evaluation_results, catching the failure the way MLflow's evaluation guide does, with except mlflow.exceptions.ModelValidationFailedException. When the model fell short, Python raised AttributeError: module 'mlflow.exceptions' has no attribute 'ModelValidationFailedException' instead of printing my message. In MLflow 3.18.0 the class lives in mlflow.models.evaluation.validation; importing it from there fixed it.
What I Was Doing
I was writing How to Set Minimum Scores for a Model with mlflow.validate_evaluation_results, and I followed the Model Validation example of MLflow's evaluation guide, which ends with except mlflow.exceptions.ModelValidationFailedException as e:. The model is a logistic regression on scikit-learn's breast cancer data, and the thresholds ask for 0.95 accuracy and 0.95 recall:
import mlflow
from mlflow.models import MetricThreshold
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
mlflow.set_tracking_uri("http://127.0.0.1:8080")
mlflow.set_experiment("thresholds-demo")
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
y = y == 0 # True when the tumour is malignant
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
model = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
with mlflow.start_run(run_name="check-thresholds"):
info = mlflow.sklearn.log_model(model, name="model", input_example=X_train.head(3))
result = mlflow.models.evaluate(
info.model_uri,
X_test.assign(malignant=y_test),
targets="malignant",
model_type="classifier",
)
thresholds = {
"accuracy_score": MetricThreshold(threshold=0.95, greater_is_better=True),
"recall_score": MetricThreshold(threshold=0.95, greater_is_better=True),
}
try:
mlflow.validate_evaluation_results(
candidate_result=result,
validation_thresholds=thresholds,
)
print("Model meets all thresholds")
except mlflow.exceptions.ModelValidationFailedException as e:
print(f"Validation failed: {e}")
uv run python thresholds.py
What I Expected
Validation failed: and the reason: this model's recall is 0.9434, below 0.95.
What I Got
Two tracebacks. The first is the validation failure I wanted to catch; the second, raised while Python handled it, is the AttributeError:
2026/11/21 17:53:50 INFO mlflow.models.evaluation.validation: Validating candidate model metrics against baseline
Traceback (most recent call last):
File "/Users/you/source/mlflow-demo/thresholds.py", line 32, in <module>
mlflow.validate_evaluation_results(
...
mlflow.models.evaluation.validation.ModelValidationFailedException: Metric recall_score value threshold check failed: candidate model recall_score = 0.9433962264150944, recall_score threshold = 0.95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/you/source/mlflow-demo/thresholds.py", line 37, in <module>
except mlflow.exceptions.ModelValidationFailedException as e:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'mlflow.exceptions' has no attribute 'ModelValidationFailedException'

What I Checked
Where the class really is. The first traceback names it in full: mlflow.models.evaluation.validation.ModelValidationFailedException. MLflow 3.18.0's mlflow/exceptions.py defines MlflowException and others, but not this one, and mlflow/models/__init__.py doesn't export it either.
What the reference says. mlflow.validate_evaluation_results says "If the candidate results do not meet the validation thresholds, an ModelValidationFailedException will be raised", without a module. Its signature does name the module of its thresholds: validation_thresholds: dict[str, mlflow.models.evaluation.validation.MetricThreshold].
Why it only failed now. Python looks up the class in an except clause only when an exception reaches that clause. With every threshold met, the script prints Model meets all thresholds and never touches the missing name; the mistake waits for the first model that fails.
Why It Happens
The guide's example names the exception through mlflow.exceptions, where MLflow 3.18.0 doesn't define it. The class is defined, and raised, in mlflow.models.evaluation.validation, next to MetricThreshold.
The Fix
Import the class from the module that defines it, and catch it by name:

from mlflow.models import MetricThreshold
from mlflow.models.evaluation.validation import ModelValidationFailedException
# ...
try:
mlflow.validate_evaluation_results(
candidate_result=result,
validation_thresholds=thresholds,
)
print("Model meets all thresholds")
except ModelValidationFailedException as e:
print(f"Validation failed: {e}")
After the Fix
uv run python thresholds.py
🏃 View run check-thresholds at: http://127.0.0.1:8080/#/experiments/10/runs/f666a5485585448cae6360c910122fc5
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/10
2026/11/21 18:58:12 INFO mlflow.models.evaluation.validation: Validating candidate model metrics against baseline
Validation failed: Metric recall_score value threshold check failed: candidate model recall_score = 0.9433962264150944, recall_score threshold = 0.95.

The failure is caught, and the script prints which threshold the model missed.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
| Testing the script only with a model that passes | The except clause never runs, so the mistake stays hidden | Also run it with a threshold the model misses |
except Exception to get past it | Every other error is caught as a failed gate too | Catch ModelValidationFailedException |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
AttributeError: module 'mlflow.exceptions' has no attribute 'ModelValidationFailedException' | The class named through mlflow.exceptions | Import it from mlflow.models.evaluation.validation |
MlflowException: The validation thresholds argument must be a dictionary that maps strings to MetricThreshold objects. | Thresholds that aren't MetricThreshold objects | Wrap each one in MetricThreshold(...) |
FAQ
Can I catch MlflowException instead?
It works, since ModelValidationFailedException is a subclass of MlflowException, but it also catches MLflow's other errors, such as thresholds of the wrong type. Catching the exact class keeps a failed gate apart from a broken one.
Does the gate still fail without the except clause?
Yes: validate_evaluation_results raises, and an uncaught exception stops the script with a traceback and exit code 1.
Conclusion
The evaluation guide's example catches mlflow.exceptions.ModelValidationFailedException, which MLflow 3.18.0 doesn't define, so the except line itself failed the first time a model missed a threshold. Importing ModelValidationFailedException from mlflow.models.evaluation.validation made the failure catchable, and the script printed which threshold was missed. I reported the guide's example to MLflow as issue #28014.
Additional Resources
- mlflow.validate_evaluation_results — the validation function and its thresholds.
- MLflow model evaluation guide — the Model Validation example.
