Fixed: MLflow Autolog's training_recall_score Is the Same as training_accuracy_score
On a yes/no model, MLflow's autologging logged a training_recall_score of 0.9044006069802731, the very same number as training_accuracy_score. That's no coincidence: without pos_label, autologging averages precision, recall and F1 over both classes, weighted by how common each is, and weighted recall always equals accuracy. With mlflow.sklearn.autolog(pos_label=1), it logs the scores of the "yes" class instead, which told a very different story: a recall of 0.27.
What I Was Doing
I was building the term deposit project of the MLflow series: a model that predicts whether a bank customer signs up for a term deposit, 1 for yes and 0 for no. Only about 1 call in 9 ends in a yes. I trained it with a GridSearchCV search and mlflow.sklearn.autolog() on MLflow 3.18.0, then opened the search's run in the MLflow UI.
What I Expected
Precision and recall that describe how well the model finds the customers who say yes, the ones the bank wants to call.
What I Got

training_recall_scorewas 0.9044006069802731, identical totraining_accuracy_scoredown to the last digit.training_precision_score, 0.8893, andtraining_f1_score, 0.8851, were close to it.
A model that finds 90% of the yeses, with a ROC AUC of only 0.83, didn't add up.
What I Checked
MLflow's reference. mlflow.sklearn.autolog has a pos_label parameter: "If given, used as the positive label to compute binary classification training metrics such as precision, recall, f1, etc." It doesn't say what happens when it isn't given.
MLflow's source. The function that computes these metrics, _get_classifier_metrics in mlflow/sklearn/utils.py, says it: "By default, when pos_label is not specified (passed in as None), we set average to weighted to compute the weighted score of these metrics. When the pos_label is specified (not None), we set average to binary."
What weighted means. scikit-learn computes the score for each class, here once for "no" and once for "yes", then averages them, weighting each by its number of rows. With 89% of the rows being "no", the average is mostly the "no" class's score.
Why It Happens
For recall, the weighted average is more than close to accuracy: it's equal. Each class's recall is the share of its rows predicted correctly. Weighting it by the class's share of all rows turns it back into a count of correct predictions over all rows, and adding the classes gives correct predictions over all rows: accuracy. So training_recall_score was accuracy under another name, and precision and F1 were dominated by the easy "no" class.
The Fix
Tell autologging which class is the positive one. In the project's tune.py, the answers are 1 for yes, so:

# Let MLflow record the search: a child run for each of the best candidates, and the
# precision, recall and F1 of the "yes" answers (1). The search only picks the settings,
# so it doesn't save the models it tries.
mlflow.sklearn.autolog(max_tuning_runs=args.max_tuning_runs, pos_label=1, log_models=False)
After the Fix
The same search, with pos_label=1:

| Metric | Weighted (before) | Yes class, pos_label=1 (after) |
|---|---|---|
training_precision_score | 0.8893 | 0.6949 |
training_recall_score | 0.9044 | 0.2699 |
training_f1_score | 0.8851 | 0.3888 |
training_accuracy_score | 0.9044 | 0.9044 |
Accuracy and the other metrics didn't change; precision, recall and F1 now describe the yes class. And they tell the real story: when the model says yes, it's right 69% of the time, but it says yes for only 27% of the customers who did sign up, at the default cut-off of 0.5. Weighted, that 27% was hidden behind the 98% of noes the model got right.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
| Reading weighted recall on an imbalanced problem as the positive class's recall | A rare class's poor recall hides behind the common class | Set pos_label |
Setting pos_label for a model that isn't a yes/no model | MLflow's reference: "If used for multi-label model, the training metrics calculation will fail and the training metrics won't be logged" | Only set it for yes/no models |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
training_recall_score equals training_accuracy_score | Weighted averages, pos_label not set | mlflow.sklearn.autolog(pos_label=1) |
| Precision and recall look too good on a rare class | The common class dominates the weighted average | Same fix |
FAQ
Which metric should I trust on an imbalanced problem?
ROC AUC judges the ranking, whatever the cut-off; precision and recall of the positive class judge the yes/no answers at your cut-off. Accuracy flatters any model that mostly answers with the common class.
Does pos_label change the model?
No. It only changes how MLflow computes the logged precision, recall and F1. The model and its predictions are the same.
Conclusion
training_recall_score equaled training_accuracy_score because autologging averages recall over both classes when pos_label isn't set, and weighted recall is accuracy. mlflow.sklearn.autolog(pos_label=1) made it log the yes class's precision, recall and F1, 0.6949, 0.2699 and 0.3888, which showed how few of the yeses the model finds at the default cut-off.
Additional Resources
- mlflow.sklearn.autolog —
pos_labeland the training metrics. - recall_score — the
averageoptions, includingweightedandbinary.
