How to scrape Google results in three lines of Python code

Here's a quick and easy way to scrape Google search engine results into a Pandas dataframe in just three lines of Python code...

How to scrape Google results in three lines of Python code
Picture by Ciao, Pexels.
3 minutes to read

EcommerceTools makes it really quick and easy to scrape Google search engine results in Python. In this simple project, we’ll use EcommerceTools to search Google for your chosen keywords, scrape the search engine results, and display the output in a Pandas dataframe.

Install EcommerceTools

First, open up a Jupyter notebook or a terminal and install the EcommerceTools Python package using the Pip Python package manager. You can do this in Jupyter by entering the command below and then pressing shift and enter to execute the cell.

!pip3 install --upgrade ecommercetools

Import packages

Next, import the seo module from ecommercetools by entering from ecommercetools import seo. This will import only the SEO module we need for this task.

from ecommercetools import seo

Scrape the Google search results

Finally, pass your search query to the seo.get_serps() function and assign the output to a variable and then print the results. The function will go to Google, search for your chosen search term, and scrape the first page of results and return them in a Pandas dataframe.

results = seo.get_serps("scrape google results")
results
title link text
0 Is it ok to scrape data from Google results? -... https://stackoverflow.com/questions/22657548/i...
1 Free Google Search scraper and SERP API · Apify https://apify.com/apify/google-search-scraper Unofficial API to scrape and export data from ...
2 How to scrape Google Search results | Apify Blog https://blog.apify.com/unofficial-google-searc... 25 Feb 2019 — Unofficial Google Search API fro...
3 How To Scrape Google With Python | Hacker Noon https://hackernoon.com/how-to-scrape-google-wi... 28 Dec 2019 — So I made my own, and here is a ...
4 Scrape 100 Google search results in 30 seconds... https://samuelschmitt.com/scrape-google-search... 30 Sept 2019 — Scrape Google Search and get th...
5 How to Scrape Google Results for Free Using Py... https://predictivehacks.com/how-to-scrape-goog... 24 Oct 2020 — scrape google results in pyhton....
6 Here's how you can scrape Google search result... https://ajaypanthagani.medium.com/heres-how-yo... 10 Sept 2019 — Here I would like to tell you a...
7 Scrape Google Search Results using Python Beau... https://www.journaldev.com/44755/scrape-google... 1 What is Web Scraping? 2 How Does Web Scrapin...
8 How to scrape Google search results using Python https://practicaldatascience.co.uk/data-scienc... 13 Mar 2021 — Scraping Google search results u...
9 Google Scraper 101 | Easy Data Technology | Sc... https://scrapingrobot.com/blog/google-scraper/ Why Scrape Google Results? — In the case of th...

Matt Clarke, Sunday, May 02, 2021

Matt Clarke Matt is a Digital Director who uses data science to help in his work. He has a Master's degree in Internet Retailing (plus two other Master's degrees in different fields) and specialises in the technical side of ecommerce and marketing.

Web Scraping in Python

Learn to retrieve and parse information from the internet using the Python library scrapy.

Start course for FREE

Comments