Skip to main content

How to Use class_weight='balanced' for Imbalanced Classes in scikit-learn

· 4 min read
Jagdish Kumawat
Founder @ Dewiride

When one class is rare, a classifier learns mostly from the common one, and misses many of the rare cases. Many scikit-learn classifiers take class_weight="balanced", which makes each class weigh the same in training, whatever its size. This guide trains the same model with and without it on data where 1 row in 10 is rare, and compares what it catches, with real output from scikit-learn 1.9.1.

Syntax​

Syntax
model = Classifier(class_weight="balanced")
PartWhat it means
ClassifierA scikit-learn classifier that takes class_weight, such as LogisticRegression or HistGradientBoostingClassifier
class_weight="balanced"Weights every row by its class's rarity, so each class counts as much as the others in total

The reference for LogisticRegression gives the formula: "The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y))." The default, None, gives every row the same weight.

Step 1: Train With and Without It​

The example makes 2,000 rows with make_classification, about 1 in 10 of them in the rare class 1, and trains a logistic regression both ways:

The class_weight.py script in a macOS-style code window: 2,000 made-up rows with weights 0.9 and 0.1, a stratified split, and a loop that trains LogisticRegression with class_weight None and balanced and prints recall and precision.

class_weight.py
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
from sklearn.model_selection import train_test_split

# 2,000 made-up rows; about 1 in 10 belongs to the rare class, 1.
X, y = make_classification(n_samples=2000, weights=[0.9, 0.1], random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=0)

for class_weight in [None, "balanced"]:
model = LogisticRegression(class_weight=class_weight).fit(X_train, y_train)
predicted = model.predict(X_test)
print(
f"class_weight={class_weight!s:<9} "
f"recall={recall_score(y_test, predicted):.3f} "
f"precision={precision_score(y_test, predicted):.3f}"
)
Terminal
uv run python class_weight.py
Output
class_weight=None recall=0.923 precision=1.000
class_weight=balanced recall=0.942 precision=0.860

The output of class_weight.py in a terminal: with class_weight None, recall 0.923 and precision 1.000; with balanced, recall 0.942 and precision 0.860.

With "balanced", the model caught more of the rare class, recall 0.923 to 0.942, and paid for it with false alarms, precision 1.000 to 0.860. Giving the rare class more weight moves the model's answers towards it: more of the real cases found, more ordinary rows flagged by mistake.

Common Mistakes​

MistakeWhat happensDo this instead
Judging the result on accuracy aloneThe extra false alarms lower accuracy, 0.992 to 0.978 here, and the gain on the rare class is hiddenCompare recall and precision, or balanced accuracy
Reading the weighted model's probabilities as true chancesThe weights push its probabilities towards the rare classCheck its calibration curve before trusting them as chances

Troubleshooting​

SymptomLikely causeFix
TypeError: GradientBoostingClassifier.__init__() got an unexpected keyword argument 'class_weight'That estimator doesn't take class weightsCheck its reference for class_weight, or use one that has it, such as HistGradientBoostingClassifier

FAQ​

Does class_weight change the data?​

No: the rows stay the same, and each one counts more or less in the model's loss.

Do the probabilities change too?​

Yes. A model trained with balanced weights gives the rare class higher probabilities than its real share: How to Plot a Calibration Curve in scikit-learn shows it on this kind of data.

Conclusion​

class_weight="balanced" gave the rare class as much weight as the common one, and the model caught more rare cases, recall 0.942 against 0.923, at the cost of more false alarms, precision 0.860 against 1.000.

Additional Resources​