Skip to main content

How to Choose the Columns of a pandas DataFrame with reindex

· 5 min read
Jagdish Kumawat
Founder @ Dewiride

Tables often have more columns than you need, in an order you didn't choose. pandas picks columns by name in two ways: frame[[...]], which needs every name to exist, and reindex(columns=[...]), which adds an empty column for a name that doesn't. This guide picks columns both ways from a small table of test scores, including one column that isn't there, with real output from pandas 3.1.1.

Syntax​

Syntax
frame[["<column>", "<column>"]]
frame.reindex(columns=["<column>", "<column>"])
PartWhat it means
frame[[...]]A new DataFrame with these columns, in this order. Every name must be a column of frame
frame.reindex(columns=[...])The same, except that a name that isn't a column becomes a new column, filled with NaN: reindex "places NA/NaN in locations having no value in the previous index"
The listThe columns to keep, in the order you want them. Columns not in the list are left out

Both return a new DataFrame and leave frame as it is.

Step 1: Pick Columns Both Ways​

The example is a table of two pupils' marks in maths and art. Create pick_columns.py, which keeps the names and art marks, then asks reindex for a music column that doesn't exist:

The pick_columns.py script in a macOS-style code window: it builds a DataFrame with name, math and art columns for Ana and Ben, prints scores[["name", "art"]], and prints scores.reindex(columns=["name", "art", "music"]).

pick_columns.py
import pandas as pd

scores = pd.DataFrame({"name": ["Ana", "Ben"], "math": [91, 78], "art": [85, 88]})

print(scores[["name", "art"]])
print(scores.reindex(columns=["name", "art", "music"]))
Terminal
uv run python pick_columns.py
Output
name art
0 Ana 85
1 Ben 88
name art music
0 Ana 85 NaN
1 Ben 88 NaN

A terminal running uv run python pick_columns.py: a table with name and art for Ana 85 and Ben 88, then the same table with a third column, music, holding NaN for both.

  • scores[["name", "art"]] kept two columns and left out math.
  • reindex did the same and added music, empty, because the table has no such column. NaN means "no value".

Step 2: See What Brackets Do with a Missing Column​

Create missing_column.py, which asks the brackets for the same three columns:

missing_column.py
import pandas as pd

scores = pd.DataFrame({"name": ["Ana", "Ben"], "math": [91, 78], "art": [85, 88]})

print(scores[["name", "art", "music"]])
Terminal
uv run python missing_column.py
Output (last line)
KeyError: "['music'] not in index"

The brackets refuse a name that isn't a column. That's useful when a missing column means something went wrong; reindex is the choice when a column may legitimately be missing, such as a value that only some rows of your data ever had.

Common Mistakes​

MistakeWhat happensDo this instead
Single brackets around two names, scores["name", "art"]KeyError: ('name', 'art'): pandas looks for one column named by the pairUse double brackets: scores[["name", "art"]]
Brackets for a column that may be missingKeyError: "['music'] not in index"Use reindex(columns=[...])
A typo in a reindex nameNo error: you get an extra empty column and lose the real oneCheck the result's columns, or use brackets when every name must exist

Troubleshooting​

SymptomLikely causeFix
A whole column of NaN after reindexThe name isn't a column of the table, often a typoCompare it with list(frame.columns)
KeyError: "[...] not in index"A name in the brackets isn't a columnFix the name, or use reindex
The original table still has every columnBoth ways return a new DataFrameAssign the result: scores = scores[["name", "art"]]

FAQ​

Can reindex fill missing columns with something other than NaN?​

Yes: fill_value sets it, such as reindex(columns=[...], fill_value=0). The default is NaN.

Does the order of the list matter?​

Yes. Both ways return the columns in the order you list them, so the same call also reorders columns.

How do I drop a few columns instead of listing the ones to keep?​

frame.drop(columns=["math"]) removes the named columns and keeps the rest.

Conclusion​

scores[["name", "art"]] kept the columns that exist, in the order given, and raised KeyError for one that doesn't. scores.reindex(columns=["name", "art", "music"]) kept the same columns and added the missing one as an empty NaN column, so code that expects a set of columns always gets them.

Additional Resources​