Skip to main content

How to Transform Only Some Columns with ColumnTransformer in scikit-learn

· 4 min read
Jagdish Kumawat
Founder @ Dewiride

Real tables mix columns that need work, such as words to encode, with columns that are fine as they are, such as counts. ColumnTransformer applies each step to the columns you name and, with remainder="passthrough", keeps the rest untouched. This guide one-hot encodes the city of three flats and passes their number of rooms through, with real output from scikit-learn 1.9.1.

Syntax​

Syntax
prepare = ColumnTransformer(
[(name, transformer, columns)],
remainder="passthrough",
)
table = prepare.fit_transform(frame)
PartWhat it means
(name, transformer, columns)One step: a name you choose, the transformer to apply, and the column names it applies to. List as many steps as you need
remainder="passthrough"Keep every column no step names, unchanged. The default, "drop", leaves them out
fit_transform(frame)Learns what each transformer needs, then returns one table with all the results side by side

Step 1: Encode One Column, Keep the Other​

Create column_transformer.py:

The column_transformer.py script in a macOS-style code window: it builds a flats DataFrame with city Pune, Delhi, Pune and rooms 2, 3, 1, then a ColumnTransformer that one-hot encodes the city column with remainder passthrough, and prints the feature names and the transformed table.

column_transformer.py
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

flats = pd.DataFrame({"city": ["Pune", "Delhi", "Pune"], "rooms": [2, 3, 1]})

prepare = ColumnTransformer(
[("city", OneHotEncoder(sparse_output=False), ["city"])],
remainder="passthrough",
)
table = prepare.fit_transform(flats)

print(prepare.get_feature_names_out())
print(table)
Terminal
uv run python column_transformer.py
Output
['city__city_Delhi' 'city__city_Pune' 'remainder__rooms']
[[0. 1. 2.]
[1. 0. 3.]
[0. 1. 1.]]

A terminal running uv run python column_transformer.py: the names city__city_Delhi, city__city_Pune and remainder__rooms, then three rows such as 0, 1, 2 for the first flat in Pune with 2 rooms.

Step 2: Read the Columns​

  • city__city_Delhi and city__city_Pune came from the step named city: the encoder's own names, city_Delhi and city_Pune, after the step's name and two underscores.
  • remainder__rooms is the rooms column, passed through untouched, after the transformed ones.

So the first flat, in Pune with 2 rooms, became [0. 1. 2.]. How to One-Hot Encode a Column with OneHotEncoder in scikit-learn explains the encoder's part.

Common Mistakes​

MistakeWhat happensDo this instead
Leaving out remainderThe default, "drop", removes rooms: the output only has city__city_Delhi and city__city_Puneremainder="passthrough" for the columns you want to keep
Naming a column that isn't in the tableValueError: Some column names are not columns of the dataframe: {'town'}Check frame.columns
Expecting the original column orderTransformed columns come first, the remainder lastRead the order from get_feature_names_out()

Troubleshooting​

SymptomLikely causeFix
A column is missing from the outputremainder is "drop", the defaultSet remainder="passthrough"
The output is a sparse matrixA step returned sparse output and the whole result is less than 30% non-zero, the default sparse_thresholdsparse_output=False on the encoder, or sparse_threshold=0

FAQ​

Can each column get a different step?​

Yes: list one (name, transformer, columns) per step, such as an encoder for the text columns and a scaler for the number columns.

How do I use it with a model?​

Put it first in a Pipeline, so the same preparation runs on training data and on every prediction: How to Chain Preprocessing and a Model with a Pipeline in scikit-learn.

Conclusion​

ColumnTransformer([("city", OneHotEncoder(sparse_output=False), ["city"])], remainder="passthrough") encoded the city and kept the rooms, naming each output column after its step. Set remainder="passthrough" to keep the columns you don't transform, and read the column order from get_feature_names_out().

Additional Resources​