How to Calculate ROC AUC in scikit-learn
ROC AUC measures how well a model ranks: the chance that it gives a randomly picked positive row a higher score than a randomly picked negative one. 1.0 is a perfect ranking and 0.5 is no better than a coin. Because it judges the ranking, not a yes/no cut-off, it suits problems where positives are rare. scikit-learn computes it with roc_auc_score; this guide scores a tiny example and a real model, with real output from scikit-learn 1.9.1.
Syntax
score = roc_auc_score(y_true, y_score)
| Part | What it means |
|---|---|
y_true | The true labels, such as 0 and 1 |
y_score | One score per row, higher meaning "more likely positive". For a scikit-learn classifier, the probability of the positive class: model.predict_proba(X)[:, 1] |
For a yes/no problem, the documentation says y_score "corresponds to an array of shape (n_samples,)" whose probability estimates are those "of the class with the greater label": column 1 of predict_proba.
Step 1: Score a Tiny Example
Four rows, two of each class, with the scores a model might give them. Create roc_auc.py:

from sklearn.metrics import roc_auc_score
y_true = [0, 0, 1, 1]
y_score = [0.1, 0.4, 0.35, 0.8]
print(roc_auc_score(y_true, y_score))
uv run python roc_auc.py
0.75
There are 2 × 2 = 4 positive-negative pairs. The model ranked three of them right, and one wrong: the positive scored 0.35, below the negative's 0.4. Three out of four is 0.75.
Step 2: Score a Real Model
Create roc_auc_model.py, which trains a gradient boosting classifier on scikit-learn's breast cancer data and scores its test set twice: from probabilities, and from the 0/1 labels predict returns:

from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=0, stratify=y
)
model = HistGradientBoostingClassifier(random_state=0).fit(X_train, y_train)
print("From probabilities:", round(roc_auc_score(y_test, model.predict_proba(X_test)[:, 1]), 4))
print("From labels: ", round(roc_auc_score(y_test, model.predict(X_test)), 4))
uv run python roc_auc_model.py
From probabilities: 0.9866
From labels: 0.9306

From probabilities, 0.9866. From labels, 0.9306: with only 0 and 1 as scores, many positive-negative pairs get the same score, each such tie counts as half right, and the finer ranking the probabilities held is lost. Always pass probabilities.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
Passing model.predict(X) | A lower score that ignores the ranking: 0.9306 instead of 0.9866 here | model.predict_proba(X)[:, 1] |
Passing all of predict_proba(X) | ValueError: y should be a 1d array, got an array of shape (169, 2) instead. | Take column 1: [:, 1] |
| A test set with only one class | UndefinedMetricWarning: Only one class is present in y_true. ROC AUC score is not defined in that case., and the result is nan | Split with stratify=y so both classes are present |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
The score is nan | Only one class in y_true | Check the labels; split with stratify |
| The score is below 0.5 | The scores point the wrong way: column 0 instead of 1, or a "lower is more likely" score | Use column 1 of predict_proba |
FAQ
Is a high ROC AUC enough?
It says the model ranks well, not that its yes/no answers are good: those depend on the cut-off you choose. Look at precision and recall at your cut-off too.
Can GridSearchCV tune for it?
Yes: GridSearchCV(..., scoring="roc_auc"): How to Tune Hyperparameters with GridSearchCV in scikit-learn.
Conclusion
roc_auc_score([0, 0, 1, 1], [0.1, 0.4, 0.35, 0.8]) returned 0.75, three of four pairs ranked right, and a real model scored 0.9866 from its probabilities but only 0.9306 from its labels. Pass column 1 of predict_proba, and make sure the test set holds both classes.
Additional Resources
- roc_auc_score — its parameters, including the binary case.
