How to use the eBay Finding API with Python

The eBay SDK allows developers to search and retrieve eBay listings using a Python API. Here's how to use it to create a dataset of Land Rover Defender listings and import them into Pandas.

How to use the eBay Finding API with Python
Picture by Jon Flobrant, Unsplash.
10 minutes to read

The eBay Finding API gives you direct access to eBay search listings using a simple SDK. This API lets you search or query eBay to fetch specific search listings for items that are active or ended, so you can analyse the data and use it in your research or models.

If you work in ecommerce and run marketplaces on eBay and other trading platforms, monitoring eBay listings can be a useful tactic. It can tell you what your rivals are selling lots of, and it can help you monitor their prices.

In this project, I’ll show you how you can use the Python eBay SDK to access the Finding API and fetch data on Land Rover Defenders currently for sale in the UK. Here’s how it’s done.

Load the packages

First, open a Jupyter notebook and import Pandas and the eBay SDK. This is an unofficial Python package that allows you to query and search the eBay Finding API and retrieve items from eBay search results. You can install this by entering pip3 install ebaysdk.

To give better visibility of the data, I’ve used a couple of Pandas set_option() functions to make the columns wider and show more rows than the default 100.

!pip3 install ebaysdk
import pandas as pd
from ebaysdk.finding import Connection as Finding
from ebaysdk.exception import ConnectionError
pd.set_option('max_colwidth', 1000)
pd.set_option('max_rows', 1000)

Create an eBay API key

Next, go to the eBay Developers Program site and create an application and get an application key. You’ll use this to authenticate your application with the eBay API. I’ve assigned mine to a constant that I can use throughout my code.

APPLICATION_ID = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

Create a function to search the eBay Finding API

Now we’ll make a series of functions to handle the tasks of connecting to the Finding API, finding the total number of pages for our given search, paginating through the results and returning the output in a Pandas dataframe.

First, we’ll make a get_results() function to pass our search query to the eBay API. This will use our APPLICATION_ID and will take a dictionary called payload that contains our specific search query. It will return a response dictionary if it works, or an exception if it fails.

def get_results(payload):
    try:
        api = Finding(siteid='EBAY-GB', appid=APPLICATION_ID, config_file=None)
        response = api.execute('findItemsAdvanced', payload)
        return response.dict()
    except ConnectionError as e:
        print(e)
        print(e.response.dict())

Here’s the payload dictionary we’re going to pass to eBay. This will search for the keyword “Defender” in category 29748, which contains Land Rovers, will restrict the search to items listed in Great Britain, and will return them in order of the newest listings first.

payload = {
        'keywords': 'Defender', 
        'categoryId': ['29748'],
        'itemFilter': [
            {'name': 'LocatedIn', 'value': 'GB'},
        ],
        'sortOrder': 'StartTimeNewest',
}

results = get_results(payload)

Since the eBay API only returns 100 results at a time, we’ll need to paginate through these in order to get all the listings for Land Rover Defenders. The dictionary returned contains a paginationOutput section with a value called totalPages holding the number of pages we need to retrieve, so we’ll grab that with the get_total_pages() function.

def get_total_pages(results):
    if results:
        return int(results.get('paginationOutput').get('totalPages'))
    else:
        return

Get the paginated results

Now we’ll put together those functions and paginate through the results. First, we’ll pass our initial payload to get_results(), then we’ll find the number of total_pages in the results. We’ll then extract the list of items in the searchResult part of the dictionary and assign it to items_list. By default, this contains the first 100 items.

Then we’ll create a while loop to fetch the data from the other pages. We’ll increment the pageNumber value and append a paginationInput component to the payload and then re-run it. Each time we get a new set of items, we’ll add the list to the initial items_list list using extend().

Finally, we’ll create a Pandas dataframe in which to store the results, then we’ll parse the output of each item and add it to the dataframe as a new row, then we’ll return the finished dataframe containing each listing from every page in the results.

def search_ebay(payload):
    
    results = get_results(payload)
    total_pages = get_total_pages(results)
    items_list = results['searchResult']['item']
        
    i = 2
    while(i <= total_pages):
        payload['paginationInput'] = {'entriesPerPage': 100, 'pageNumber': i}        
        results = get_results(payload)
        items_list.extend(results['searchResult']['item'])
        i += 1
        
    df_items = pd.DataFrame(columns=['itemId', 'title', 'viewItemURL', 'galleryURL', 'location', 'postalCode',
                                 'paymentMethod''listingType', 'bestOfferEnabled', 'buyItNowAvailable',
                                 'currentPrice', 'bidCount', 'sellingState'])

    for item in items_list:
        row = {
            'itemId': item.get('itemId'),
            'title': item.get('title'),
            'viewItemURL': item.get('viewItemURL'),
            'galleryURL': item.get('galleryURL'),
            'location': item.get('location'),
            'postalCode': item.get('postalCode'),
            'paymentMethod': item.get('paymentMethod'),        
            'listingType': item.get('listingInfo').get('listingType'),
            'bestOfferEnabled': item.get('listingInfo').get('bestOfferEnabled'),
            'buyItNowAvailable': item.get('listingInfo').get('buyItNowAvailable'),
            'currentPrice': item.get('sellingStatus').get('currentPrice').get('value'),
            'bidCount': item.get('bidCount'),
            'sellingState': item.get('sellingState'),
        }

        df_items = df_items.append(row, ignore_index=True)

    return df_items

Run the code

Now that’s all done, we can run the search_ebay() function, pass in our payload and leave it for a minute. It will use the Finding API to run our search, and will fetch the items from each page and append them to the dataframe to give us the data shown below.

df_items = search_ebay(payload)
df_items.head()
itemId title viewItemURL galleryURL location postalCode paymentMethodlistingType bestOfferEnabled buyItNowAvailable currentPrice bidCount sellingState listingType paymentMethod
0 333886487622 2002 Land Rover Defender Hard Top Td5 Four Wheel Drive Diesel Manual https://www.ebay.co.uk/itm/2002-Land-Rover-Defender-Hard-Top-Td5-Four-Wheel-Drive-Diesel-Manual -/333886487622 https://thumbs3.ebaystatic.com/m/mObPY7ibKNDXp5mrZ60ZXkQ/140.jpg Gloucester,United Kingdom GL194JL NaN false false 16250.0 None None Classified None
1 203275438277 £5000 dealer fit options, 22" alloys, wide arch kit, side steps, NAS rear lights https://www.ebay.co.uk/itm/5000-dealer-fit-options-22-alloys-wide-arch-kit-side-steps-NAS-rear -lights -/203275438277 https://thumbs2.ebaystatic.com/m/mXcckbdMBkmWL6Teok8QICA/140.jpg Corsham,United Kingdom SN138AE NaN false false 66990.0 None None Classified None
2 154326655603 2015/64 LAND ROVER DEFENDER 90 2.2 TDCI XS STATION WAGON ONLY 29 DELIVERY MILES! https://www.ebay.co. uk/itm/2015-64-LAND-ROVER-DEFENDER-90-2-2-TDCI-XS-STATION-WAGON-ONLY-29 -DELIVERY -MILES-/154326655603 https://thumbs4.ebaystatic.com/m/m1zqIxReQH3fiA-E1g0lopg/140.jpgh Congleton,United Kingdom CW123DS NaN false false 59995.0 None None Classified None
df_items.shape
(628, 14)
df_items[['title', 'currentPrice', 'location']].head()
title currentPrice location
0 2002 Land Rover Defender Hard Top Td5 Four Wheel Drive Diesel Manual 16250.0 Gloucester,United Kingdom
1 £5000 dealer fit options, 22" alloys, wide arch kit, side steps, NAS rear lights 66990.0 Corsham,United Kingdom
2 2015/64 LAND ROVER DEFENDER 90 2.2 TDCI XS STATION WAGON ONLY 29 DELIVERY MILES! 59995.0 Congleton,United Kingdom
3 Land Rover Defender 3.0 DIESEL HSE DYNAMIC X SPORT BY RACELINE REDEFINED 82950.0 Corsham,United Kingdom
4 Land Rover 90 XS 2.2TD 170bhp Sports Defender By Redefined 44950.0 Corsham,United Kingdom

Matt Clarke, Sunday, June 13, 2021

Matt Clarke Matt is a Digital Director who uses data science to help in his work. He has a Master's degree in Internet Retailing (plus two other Master's degrees in different fields) and specialises in the technical side of ecommerce and marketing.

Marketing Analytics in Spreadsheets

Learn how to ensure clean data entry and build dynamic dashboards to display your marketing data.

Start course for FREE

Comments