How to identify striking distance keywords with Python

Learn how to find striking distance keywords in Google Search Console API data with Python and improve your search engine rankings.

How to identify striking distance keywords with Python
Picture by Picography, Pexels.
13 minutes to read

Striking distance keywords are those which appear just off the bottom of the first page of search engine results. Keywords that appear on the first page have the greatest visibility and the highest search volume, making them more likely to be clicked, so a key objective in digital marketing is to get keywords ranking higher through search engine optimisation or SEO.

One popular SEO strategy involves identifying keywords that are within striking distance of the first page to optimise them, helping to focus and prioritise optimisation efforts, so results are generated faster and with the least amount of effort.

Since striking distance keywords are already ranking fairly close to the bottom of the first page, it often only takes a little more optimisation to give them a boost and move them to page one, making the technique a great way to get quick improvements in SEO.

In this project, I’ll show you how you can use Python to extract keywords and their average rankings from Google Search Console, and use Pandas to identify those which are within striking distance.

Install the packages

First, open a Jupyter notebook and install my EcommerceTools if you don’t already have it installed. You can install EcommerceTools by entering the command !pip3 install ecommercetools in a cell in your Jupyter notebook and then executing it. It will install all the dependencies you need for the project.

!pip3 install ecommercetools

We’ll be using the SEO module from EcommerceTools to fetch data from the Google Search Console account and will then display and manipulate the GSC data using Pandas, so import the pandas package and import seo from ecommercetools.

import pandas as pd
from ecommercetools import seo

Configure your Google Search Console connection

In order to authenticate with Google Search Console and extract the data from your account you will need to create a client secrets JSON keyfile. You can do this using the Google API Console. Once that’s created, download the JSON keyfile and create a variable called key to tell EcommerceTools its path.

You’ll also benefit from creating a few other variables to hold the URL of your site, and the start and end date for the period for which you want to retrieve GSC data.

key = "client-secrets.json"
site_url = "http://flyandlure.org"
start_date = '2021-01-01'
end_date = '2021-06-31'

Fetch your Google Search Console data

To fetch data from GSC using EcommerceTools you need to create a Python dictionary called a payload that contains the API parameters you want to run through the API. The payload defines the start and end date and fetches the page and query dimensions containing the URL and the search keywords the page ranks for.

By default, the GSC API will return metrics for clicks, impressions, ctr, and position. We then pass that payload to the query_google_search_console() function, along with the site_url variable and the key.

payload = {
    'startDate': start_date, 
    'endDate': end_date,
    'dimensions': ['page', 'query'],  
    'rowLimit': 10000,
    'startRow': 0
}

df = seo.query_google_search_console(key, site_url, payload)

The query_google_search_console() function will return a Pandas dataframe containing the clicks, impressions, click-through rate (CTR), and average position (or rank) for each of the queries within your selected time period. To sort these and show the biggest potential wins, I’d recommend you use the sort_values() function to show the highest volume queries at the top of your dataframe.

df.sort_values(by=['impressions', 'position'], ascending=False).head(10)
page query clicks impressions ctr position
3495 http://flyandlure.org/articles/fly_fishing_gea... waders 1 34377 0.00 6.28
3 http://flyandlure.org/articles/fly_tying/5_phe... pheasant tail nymph 189 22961 0.82 8.60
3493 http://flyandlure.org/articles/fly_fishing_gea... simms waders 1 21936 0.00 10.26
914 http://flyandlure.org/articles/lure_fishing/ho... bombarda 8 7610 0.11 28.51
1223 http://flyandlure.org/articles/fly_fishing/how... buzzer fish 5 6744 0.07 5.65
124 http://flyandlure.org/articles/fly_fishing_gea... simms freestone waders 40 6724 0.59 8.39
787 http://flyandlure.org/listings/places_to_fly_f... fisherwick lakes 10 6553 0.15 12.31
3396 http://flyandlure.org/articles/fly_fishing_gea... orvis battenkill 1 6211 0.02 10.63
7 http://flyandlure.org/articles/tagged/29/sunray sunray fly lines 149 5711 2.61 7.16
842 http://flyandlure.org/listings/fly_fishing_clu... burton mutual 9 5368 0.17 7.94

Identify striking distance keywords

Now we’ll create a little helper function to filter the above dataframe and show only the site’s striking distance keywords. There’s no formal definition of what a striking distance keyword really is.

Some people view it as any keyword that ranks on page two or three of search engine results, but you might want to define tighter criteria, such as 10 to 14 on larger sites, or those where search engine optimisation is constant.

We’ll pass in the dataframe of GSC query metrics and first filter it, so it only shows those queries that exceed a minimum volume of impressions, as this will help us focus on the keywords likely to generate the biggest search traffic improvements. Then, we’ll filter the dataframe again, so it shows only the keywords in the ranking range we want to examine.

def striking_distance_keywords(df, min_imps=1000, min_rank=10, max_rank=30):
    """Return a dataframe containing only striking distance keywords.
    
    Args:
        df (dataframe): Pandas dataframe containing query, impressions and position
        min_imps (int, optional, default=1000): Optional minimum volume of impressions
        min_rank (int, optional, default=10): Minimum search rank
        max_rank (int, optional, default=10): Maximum search rank
        
    Returns:
        df (dataframe): Filtered dataframe containing only the keywords within striking distance
    """
    
    df = df[df['impressions'] >= min_imps]
    df = df[(df['position'] > min_rank) & (df['position'] < max_rank)]
    return df.sort_values(by='position', ascending=True)

Now we can run our striking_distance_keywords() function using the dataframe of page and query data we retrieved from the Google Search Console API. I’ve set the function to require a minimum of 10 impressions, and have defined a striking distance keyword as one that appears between position 10 and 30.

df_striking_distance = striking_distance_keywords(df, min_imps=10, min_rank=10, max_rank=30)
df_striking_distance.head(10)
page query clicks impressions ctr position
3993 http://flyandlure.org/listings/fly_fishing_clu... monikie angling club 1 525 0.19 10.01
3426 http://flyandlure.org/articles/fly_fishing_gea... orvis sling bag 1 103 0.97 10.01
2296 http://flyandlure.org/articles/fly_fishing_gea... loop q reel spare spool 2 81 2.47 10.02
1632 http://flyandlure.org/listings/places_to_fly_f... chatton trout fishery 4 2504 0.16 10.02
895 http://flyandlure.org/articles/fly_fishing_gea... loop cross sx for sale 8 187 4.28 10.03
2036 http://flyandlure.org/listings/places_to_fly_f... loch migdale fishing 3 125 2.40 10.03
4277 http://flyandlure.org/listings/places_to_fly_f... ravensthorpe fly fishing 1 30 3.33 10.03
6455 http://flyandlure.org/articles/fly_fishing/cho... what weight fly rod for pike 0 27 0.00 10.04
2026 http://flyandlure.org/listings/places_to_fly_f... arranview fishery moscow 3 388 0.77 10.04
1855 http://flyandlure.org/articles/fly_fishing_gea... best fly fishing stripping basket 3 70 4.29 10.04

Obviously, since site rankings change all the time, you need to repeat this process regularly and obtain the current striking distance keywords, make your optimisations, and then repeat the process. Doing so should see your average search positions and your traffic improve.

Improving rankings for striking distance keywords

There are several things you can try to improve the search engine rankings for striking distance keywords, but one of the most popular methods is to improve internal links for these phrases by identifying other pages on your site that include these keywords or related ones, and then adding internal links to the page shown in the striking distance keywords report we created above.

A quick way to do this is to take each of your target keywords returned in the striking distance keyword report and perform a site: search for your domain on Google with the keyword afterwards in double quotes, for example: site:flyandlure.org "simms waders".

This will search pages Google has indexed and return any pages where an exact match is found for the chosen keyword phrase. All you need to do then is go to the page returned, edit the content, and add internal links to the target page from each page identified.

seo.get_serps("site:flyandlure.org \"simms waders\"")
title link text
0 Simms Freestone Waders review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 28 Dec 2018 — If your feet are bigger than an ...
1 Simms Freestone StreamTread Wading Boots revie... http://flyandlure.org/articles/fly_fishing_gea... 28 Dec 2018 — I'm a UK size 11 and bought boot...
seo.get_serps("site:flyandlure.org \"orvis battenkill\"")
title link text
0 Orvis Battenkill fly reel review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 30 Aug 2015 — The Orvis Battenkill fly reel is...
1 Orvis Access Mid Arbor fly reel review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 30 Jun 2015 — You're not paying a load of mone...
2 Orvis reviews | Fly&Lure http://flyandlure.org/articles/tagged/15/orvis Orvis Battenkill fly reel review. The Orvis Ba...
3 Orvis Superfine Touch fly rod review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 1 Jan 2016 — The Orvis Battenkill is a great m...
4 How to choose a fly reel | Fly&Lure http://flyandlure.org/articles/fly_fishing/how... 12 Jan 2019 — If you're a river angler, the Or...
5 Fly reel reviews | Fly&Lure http://flyandlure.org/articles/tagged/6/fly_reels The Airflo Xceed fly reel is made from machine...
6 Airflo Xceed fly reel review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 20 Sept 2015 — Are they heavy? My 5/6 Airflo X...
7 Hardy Zephrus FWS fly rod review | Fly&Lure http://flyandlure.org/articles/fly_fishing_gea... 20 Aug 2016 — You need to carefully match it w...
8 How to land a fish on a fly rod | Fly&Lure http://flyandlure.org/articles/fly_fishing/how... 18 Nov 2017 — On click pawl reels like the Orv...

Another effective technique is to identify keyword opportunities present in pages for which you are already ranking.

For example, if your site is already receiving traffic for the organic keywords such as “fly reels”, but that keyword phrase is missing from both the title and meta description, then adding it can increase visibility in search engine results, generating extra clicks.

Matt Clarke, Saturday, August 07, 2021

Matt Clarke Matt is a Digital Director who uses data science to help in his work. He has a Master's degree in Internet Retailing (plus two other Master's degrees in different fields) and specialises in the technical side of ecommerce and marketing.

Marketing Analytics in Spreadsheets

Learn how to ensure clean data entry and build dynamic dashboards to display your marketing data.

Start course for FREE

Comments