How to use Pandas head() and tail() to get the first and last rows

One of the first things you’ll do whenever you import a Pandas dataframe is view the data to check that it’s formatted correctly and see what you’re dealing with. It’s an important step since about 80% of what we data scientists do is, unfortunately, just cleaning and reformatting data before we can do more interesting stuff with it.

Pandas includes a few very useful functions for viewing and checking data in dataframes. In this simple tutorial we’ll be going over the head() function used for showing the first rows, the tail() function for showing the last rows, the sample() function for showing random rows, and the T or transpose function for flipping the orientation of the dataframe.

Import Pandas and load a dataframe

To get started, open a Jupyter notebook, import Pandas and Numpy using the import pandas as pd and import numpy as np naming conventions, and create a dummy dataset containing some random data. The Pandas shape method can be used to return a tuple indicating how many rows and columns are present.

import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(1000, 4), columns=list('ABCD'))
df.shape

(1000, 4)

Using head() to view the first rows of a DataFrame

The Pandas head() function is used to return the first rows from a dataframe. By default, it returns the first 5 rows, so when you call head() without any arguments you’ll get 5 rows back, unless you’ve used set_option() to increase the default value.

df.head()

To return a specific number of rows, such as just the first row, you can pass an integer value to the head() function, so df.head(1) will return the first row and df.head(10) will return the first 10 rows.

df.head(1)

Using set_option() to prevent truncation

When you import Pandas it sets a number of default display options, such as the max_rows option, which defines the maximum number of rows displayed. You can find the value of the max_rows option by calling pd.get_option('max_rows'). By default, this is set to 60 rows.

pd.get_option('max_rows')

60

If you attempt to print the whole df dataframe, or call the head() function with a value that exceeds the max_rows setting, i.e. df.head(100), Pandas will return the data in a truncated view in which the first five and last five rows are shown with ellipses in the middle to denote that data is missing.

df.head(100)

If you want to override the default max_rows value you can use the Pandas set_option function to set the value to a higher number. Now you’ll see a larger number of rows. This is very useful when you just need to scan the data by eye to check that it looks right.

pd.set_option('max_rows', 100)
df.head(100)

Using tail() to view the last rows of a DataFrame

The Pandas tail() function works just like the head() function but instead shows the last rows. Calling the function with no arguments will return the last five rows in the dataframe.

df.tail()

As with head(), you can also pass an integer value to the tail() function to return a specific number of rows from the bottom of the dataframe, so df.tail(1) will return only the last row.

df.tail(1)

Using transpose to flip the orientation of the data

The Pandas transpose function T can be used to flip the orientation of the dataframe, so the columns become rows and the rows become columns. Transposing a dataframe is an extremely useful technique for visually comparing or checking data, especially on dataframes that are wide due to the presence of lots of columns or long column values.

df.head(3).T

Using negative values in head() and tail()

More rarely, you might also see a negative value being passed to the head() or tail() functions. For example, calling df.head(-10) returns all rows apart from the first 10, while df.tail(-10) returns all rows apart from the last 10.

df.head(-10)

df.tail(-990)

Using sample() to return a random sample of rows

If you want to return a random sample of rows, you can use the Pandas sample() function. Like head() and tail(), sample() takes an optional n parameter, which specifies the number of rows to return. If you don’t specify n, it will return a single row. So, df.sample() returns a single random row, and df.sample(3) returns 3 random rows.

df.sample()

df.sample(3)

Matt Clarke, Saturday, November 26, 2022