How to analyse Google Analytics demographics and interests with GAPandas

The demographics and interests data provided in Google Analytics can be a useful way to understand who is visiting your site or purchasing your products, without the need to perform a complex and expensive demographic segmentation project that might not deliver an ROI.

While there are some limitations to the demographic data provided in Google Analytics, they are still quite useful for developing a simple overview of your customer base and can work alongside market segmentation or customer segmentation and reveal patterns of which you may not have been aware.

Having worked in businesses which have segmented their customers using expensive customer segmentation techniques such as Mosaic, that use demographic segmentation to report on the overall customer base, I’m not left convinced that these are worth spending money on, compared to what you can get for free using Google Analytics data.

If you only want an overview of customer demographics, such as the age brackets, gender, geographic location, and market interests of your customers, then the demographic and interests data in Google Analytics should be enough to give you such a high level view, for absolutely no cost.

They’re a great way to help your business owners understand who interacts with your website, how the demographics of your customer base may be changing over time, and can help you identify what products to launch, what markets to target, and what marketing activity might work best.

In this project, we’ll be using my GAPandas package to query the Google Analytics API and retrieve the demographics and interests data for the visitors to one of my (non-ecommerce) websites. This will cover the basic techniques you need to apply to understand the demographic segments of your site visitors at a high level, just as Mosaic segmentation does.

Understanding demographics and interests data in Google Analytics

Google Analytics demographics and interests data are collected using Google’s advertising cookies. Therefore, in order to see any demographic and interests data in GA, you will first need to enable a couple of optional settings.

1. Go to Google Analytics > Admin
2. Select the Google Analytics property for which you want to enable data collection
3. Set Remarketing to On
4. Set Advertising Reporting Features to On
5. Save

Once you’ve saved these settings, Google Analytics will use existing Google cookies for the DoubleClick , Android, and iOS advertising networks to capture some additional data on your site visitors and add it to your Google Analytics account in an anonymised form. Since the data is collected daily, you’ll need to leave the settings turned on for a period of time before running your analysis.

Data thresholds and anonymisation

One potential application of these data, you may be thinking, could be to capture demographic data on each of your customers to augment the customer profile you hold upon each of them. This would be a great way to let you target customers based on their demographics, and could work brilliantly alongside the aggregate demographics and interests data.

However, while this would be fascinating from a data perspective, Google doesn’t want you to be able to fetch data on the specific demographics of individual users for privacy reasons. For example, you’re unable to write an API query to fetch the userAgeBracket or userGender of each customer when you select a transactionId dimension.

In addition, if you query any data and there are few people in the dataset, which could inadvertently leak information to you about the people, Google will prevent you from seeing the data by setting a threshold on the minimum volume to anonymise the data. At the higher level queries we’re examining below, this rarely kicks in, but it might if you create more granular API queries.

Install the packages

First, open a Jupyter notebook and install the Pandas and GAPandas packages. You’ll probably have Pandas already installed, but you can install GAPandas by entering the command pip3 install gapandas in your terminal or !pip3 install gapandas in a cell in your Jupyter notebook. Any other packages we need can be installed later in our notebook.

!pip3 install gapandas

import gapandas as gp
import pandas as pd

Connect to the Google Analytics API

Next, we’ll use GAPandas to connect to the Google Analytics API. To authenticate you will need to download a client secrets JSON keyfile from the Google API Console. This gives you access to a Google Cloud Service Account, via which we can extract our Google Analytics data. There’s a step-by-step guide to creating a client secrets JSON keyfile in my other guide to using GAPandas.

service = gp.get_service('client-secrets.json')

To avoid entering the same values throughout our code, we’ll create some variables to hold the view ID for the Google Analytics property we want to query, as well as the start and end dates for the date period we want to extract via the GA API.

view = '1234567890'
start_date = '2021-07-01'
end_date = '2021-07-31'

Age bracket

The first piece of demographic data we’ll extract is the age bracket for our users, which is stored in an API dimension called userAgeBracket. To obtain this demographic data, we simply assemble a GAPandas payload dictionary containing the ga:userAgeBracket dimension and our choice of metrics, and then run the query using the run_query() function, remembering to pass in the service object and the view ID for our GA property.

As the data below show, the site has the fewest visitors in the 18-24 bracket (there is no lower age bracket in the data), and older visitors tend to dominate on this site. Although there’s no e-ecommerce data in my site’s data, I’ve included the metrics in the payload so you can apply this to analysing an e-commerce site. These will often show differences in average order value and conversion rate for each age bracket, which can reveal how well you cater for customers of different ages, or how easy or difficult they find your site to use.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:userAgeBracket',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Gender

We can repeat the process for the ga:userGender demographic data simply by adjusting the dimension we define in the payload. This reveals that the site has a strong male bias in the visitor profile. Again, there’s no e-commerce data for this site, but this also often shows big differences in customer behaviour according to gender.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:userGender',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Age and gender

To get more granular data on age and gender, you can define multiple dimensions. In the example below, I’ve included both the ga:userGender and ga:userAgeBracket dimensions, to get a more detailed breakdown of customer ages and genders. To analyse these more easily, you may wish to convert the raw metrics to percentages to better show the proportional split within each age bracket.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:userGender, ga:userAgeBracket',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)

df.sort_values(by='userAgeBracket').head(10)

Continent

You can also perform geographical segmentation using the same technique. There are various levels of dimension provided that allow you to drilldown through your data to understand where site visitors are located. From an e-commerce perspective, this can be a useful way to judge whether it may be worth considering offering delivery services to specific countries, or even offering local language content.

Do be aware, though, that all geographic data in Google Analytics should be taken with a pinch of salt. While Google has the ability to geolocate users, it would appear that these data may often be based upon the location of the network through which you are routed, so VPNs and other network systems can potentially falsify some data. However, it’s generally sufficient to give you a rough idea. Here’s the data on the ga:continent dimension, which shows most of my visitors are in Europe.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:continent',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Country

To drill down through our geographic segmentation data we can use different dimensions. By running an API query with the ga:country dimension in the payload we can see that the UK dominates the site’s traffic, followed by the US and Canada.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:country',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Region

You can also drill down to more granular levels. For example, by adding in the ga:region dimension we also get back the region (i.e. England, Wales, Scotland) for each country. To look at the regions for a specific country, you can perform a simple filter on the country column and set it to the country name.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:country, ga:region',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)

df[df['country']=='United Kingdom'].head(10)

City

To get even more detail, you can also add in the ga:city dimension and use the same approach. As you’ll notice, there’s a big chunk of (not set) data which could be anonymised to avoid revealing too much information. Looks like London and Glasgow are the main hotspots for my site.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:country, ga:region, ga:city',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)

df[df['country']=='United Kingdom'].head(10)

Designated Market Area (DMA) or Metro

You can also examine the Designated Market Area (DMA) or Metro. These are known as “media markets” or television market areas and define the TV or radio stations that a population receives. If you wanted to run TV or radio advertising, or advertise in other channels alongside your TV and radio ads, these DMAs could be very useful in helping you find the best places to advertise. On e-commerce sites, there’s sometimes a striking difference in customer behaviour between DMAs.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:country, ga:region, ga:metro',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)

df[df['country']=='United Kingdom'].sort_values(by='region', ascending=False).head(50)

Latitude and longitude

You can also get the approximate anonymised latitude and longitude coordinates for each visitor using the ga:latitude and ga:longitude dimensions. Judging from comparisons made to e-commerce data, where delivery postcodes are known, these aren’t 100% accurate as mentioned above, but they will give you a rough idea of where your visitors are physically located.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions',
    'dimensions': 'ga:latitude, ga:longitude',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)

df.head()

Distance from store

Once you have the visitor’s coordinates, you can use maths to calculate their distance from a given location using the excellent GeoPy extension. This can be a useful way to identify roughly (possibly very roughly) how many visitors you have within the radius of a particular store or event.

!pip3 install geopy

from geopy.distance import geodesic

start_latitude = 52.98
start_longitude = -3.36

df['distance'] = df.apply(lambda x: 
                          geodesic((start_latitude, start_longitude),\
                                   (x.latitude, x.longitude)).miles, axis=1)

df[df['distance']< 10].head()

Interest category

Finally, we have the interest categories, which are handled via the ga:interestOtherCategory, ga:interestAffinityCategory, and ga:interestInMarketCategory dimensions. These can tell you what interests your website visitors have, and let you drill down to get increasing levels of detail. Most of my visitors are into the outdoors, sporting goods, cars, and travel, which totally makes sense given the subject matter.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:interestOtherCategory',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Interest affinity category

The ga:interestAffinityCategory gives broader information. For my site, we can see that “Lifestyles & Hobbies/Outdoor Enthusiasts” is the top one, which again fits perfectly with the subject matter.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:interestAffinityCategory',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

In market category

Finally, there’s in-market category which is similar to the others above…

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:interestInMarketCategory',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(10)

Interest category by gender

The other thing you can do is analyse different combinations of dimensions. For example, in the code below I’m querying ga:interestOtherCategory with ga:userGender to reveal what gender differences exist within the interest categories. The topic of my site is male dominated, so the bulk of visitors are male in each category.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue, 
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:interestOtherCategory, ga:userGender',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(20)

Interest category, gender, and age bracket

You can also add in the ga:userAgeBracket to get the age brackets added in. Collectively, the various queries above give you a really good general overview of the market segments this site serves, which could be useful when communicating to key stakeholders, can help you target marketing or advertising, and can help you spot potential opportunities with products or technologies. For free, the data are really quite useful.

payload = {
    'start_date': start_date, 
    'end_date': end_date, 
    'metrics': 'ga:sessions, ga:transactions, ga:transactionRevenue,
    ga:transactionsPerSession, ga:revenuePerTransaction',
    'dimensions': 'ga:interestOtherCategory, 
    ga:userGender, ga:userAgeBracket',
    'sort': '-ga:sessions'
}

df = gp.run_query(service, view, payload)
df.head(20)

Matt Clarke, Tuesday, August 10, 2021