How to Calculate Log Loss in scikit-learn
Accuracy, precision and recall only look at a classifier's final answers. Log loss scores the probabilities behind them: it's small when the model is confident and right, and it grows fast when the model is confident and wrong. This guide scores three sets of probabilities for the same four cases, with real output from scikit-learn 1.9.1.
Syntax
from sklearn.metrics import log_loss
loss = log_loss(y_true, y_proba)
| Part | What it means |
|---|---|
y_true | The true classes |
y_proba | The probabilities the model gave each class, such as the output of predict_proba: one row per case, one column per class |
loss | The average loss per case: 0 is perfect, lower is better |
The reference for log_loss describes y_proba as "Predicted probabilities, as returned by a classifier’s predict_proba method", and notes "Added in version 1.9: y_pred was renamed to y_proba."
Step 1: Score Three Sets of Probabilities
Four cases, two of each class, and the probabilities from three imaginary models: one confident and right, one that always says 50/50, and one confident and wrong:

from sklearn.metrics import log_loss
# Four cases, and three models' probabilities for them: [chance of 0, chance of 1].
actual = [1, 0, 1, 0]
models = {
"confident and right": [[0.1, 0.9], [0.9, 0.1], [0.2, 0.8], [0.8, 0.2]],
"unsure": [[0.5, 0.5], [0.5, 0.5], [0.5, 0.5], [0.5, 0.5]],
"confident and wrong": [[0.9, 0.1], [0.1, 0.9], [0.8, 0.2], [0.2, 0.8]],
}
for name, probabilities in models.items():
print(f"{name:<20} log loss = {log_loss(actual, probabilities):.4f}")
uv run python log_loss_demo.py
confident and right log loss = 0.1643
unsure log loss = 0.6931
confident and wrong log loss = 1.9560

- 0.1643 for the confident, right model.
- 0.6931 for the model that always says 50/50: that's the natural log of 2, the score of a coin toss on two classes, a useful line to compare against.
- 1.9560 for the confident, wrong model: twelve times the first, for probabilities that are just as confident.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
| Passing the predicted classes instead of the probabilities | Each answer counts as 100% sure: one mistake in four cases gave a loss of 9.01 | Pass predict_proba's output |
log_loss(y_true=..., y_pred=...) | FutureWarning: y_pred was renamed to y_proba in version 1.9 and will be removed in 1.11 | Pass the probabilities second, or as y_proba= |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
FutureWarning: y_pred was renamed to y_proba in version 1.9 and will be removed in 1.11. Use y_proba instead. | The old keyword name | y_proba= |
| A loss far above 0.69 | Confident wrong answers, or classes passed instead of probabilities | Look at the cases with the biggest error |
FAQ
Is lower log loss always better?
Yes: 0 means the model gave every true class a probability of 1. Compare it with 0.69, the coin toss on two classes, to see whether a model's probabilities are worth anything.
Why use log loss if I already have accuracy?
Accuracy can't tell a 51% guess from a 99% one. Log loss can, which matters when you use the probabilities themselves, for example to rank cases or to estimate how many will happen.
Conclusion
log_loss gave 0.1643 to confident right probabilities, 0.6931 to a coin toss and 1.9560 to confident wrong ones: it rewards probabilities that are both sure and right.
Additional Resources
- sklearn.metrics.log_loss — the formula and the
y_probaparameter.
