How to Tune Hyperparameters with GridSearchCV in scikit-learn
A model's settings, its hyperparameters, change how well it learns, and the best values are rarely obvious. GridSearchCV tries every combination you list, scores each one with cross-validation, and keeps the best. This guide tunes the number of neighbors of a k-nearest-neighbors classifier on the iris flowers, with real output from scikit-learn 1.9.1.
Syntax
search = GridSearchCV(estimator, param_grid, cv=folds, scoring=metric)
search.fit(X, y)
| Part | What it means |
|---|---|
estimator | The model to tune, created with the settings you're not tuning |
param_grid | A dictionary from setting name to the values to try. Every combination is one candidate |
cv=folds | How many parts to split the data into for cross-validation: each candidate is trained on all parts but one and scored on the one left out, once per part |
scoring=metric (optional) | What "best" means, such as "accuracy" or "roc_auc". By default, the model's own score method |
search.fit(X, y) | Runs the whole search, then trains the best candidate again on all of X and y |
After fit, best_params_ holds the winning settings, best_score_ its average cross-validation score, cv_results_ the results of every candidate, and best_estimator_ the winning model, ready to use.
Step 1: Run a Search
The example tries four values of n_neighbors, how many nearby flowers a k-nearest-neighbors classifier looks at before it decides, with 5-fold cross-validation: 4 candidates × 5 folds = 20 fits. Create grid_search.py:

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
search = GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [1, 5, 15, 45]}, cv=5)
search.fit(X, y)
print("Best settings:", search.best_params_)
print("Best score: ", round(search.best_score_, 4))
results = pd.DataFrame(search.cv_results_)
print(results[["param_n_neighbors", "mean_test_score", "rank_test_score"]].to_string(index=False))
uv run python grid_search.py
Best settings: {'n_neighbors': 5}
Best score: 0.9733
param_n_neighbors mean_test_score rank_test_score
1 0.960000 3
5 0.973333 1
15 0.966667 2
45 0.940000 4

Step 2: Read the Results
best_params_,{'n_neighbors': 5}, is the candidate with the highest average score.best_score_, 0.9733, is that average: the share of flowers classified correctly, averaged over the 5 folds.cv_results_has a row per candidate.mean_test_scoreis each one's average andrank_test_scoreorders them, 1 being the best. Looking only at too few neighbors (1) or too many (45) scored worse than 5 or 15.
The scores here are on the same data the search chose with, so they flatter the winner a little. To know how the chosen model does on new data, keep a test set out of the search, then score search.best_estimator_ on it once: How to Split Data into Training and Test Sets with scikit-learn.
Common Mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
Misspelling a setting in param_grid, "n_neighbours" | ValueError: Invalid parameter 'n_neighbours' for estimator KNeighborsClassifier(). Valid parameters are: [...] | Copy the name from the error's list, or from get_params() |
| A huge grid | Every combination is trained once per fold: 5 values × 5 values × 5 folds is 125 fits | Start coarse, then search near the best values |
Judging the final model by best_score_ | It's the score the search optimized, so it's optimistic | Score best_estimator_ on a test set kept out of the search |
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| The search takes very long | Too many candidates, or a slow model | Fewer values, fewer folds, or n_jobs=-1 to use every CPU core |
| Several candidates share rank 1 | Their average scores are equal | Any of them is fine; prefer the simpler one |
FAQ
Is GridSearchCV the same as cross-validation?
It uses cross-validation, once per candidate. For classifiers, cv=5 splits with StratifiedKFold, which keeps the share of each class equal in every fold.
How do I tune a model inside a pipeline?
Name the setting with the step's name, two underscores and the setting: How to Set the Parameters of a Pipeline Step in scikit-learn.
Can MLflow record the search?
Yes, with one line: How to Log a GridSearchCV Search in MLflow with Parent and Child Runs.
Conclusion
GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [1, 5, 15, 45]}, cv=5) tried four candidates and picked n_neighbors=5, with an average accuracy of 0.9733. Read best_params_ for the winner and cv_results_ for every candidate, and judge the final model on data the search never saw.
Additional Resources
- GridSearchCV — its parameters and attributes.
- Tuning the hyper-parameters of an estimator — scikit-learn's guide to searching.
