How to export Zendesk tickets into Pandas using Zenpy

Zenpy is an unofficial Zendesk API for Python that allows you to export and update tickets. Here's how to use it.

How to export Zendesk tickets into Pandas using Zenpy
Picture by Mikhail Nilov, Pexels.
8 minutes to read

The Zendesk customer service platform is widely used by ecommerce businesses, but its functionality for analysing ticket trends and automatically classifying them is somewhat limited. In many cases, you might want to export your Zendesk data and use Pandas or machine learning techniques to analyse it.

In this project, I’ll show how you can use Zenpy, an unofficial Zendesk API implementation for Python to export your Zendesk tickets using Python and load them into Pandas.

If you want to classify your support tickets using zero shot learning or use Naive Bayes models for ticket classification, building your dataset is the vital first step. Here’s how it’s done.

Load the packages

First, open a Jupyter notebook and import the packages below. We’ll need just two packages: Pandas and Zenpy. Zenpy is an unofficial implementation of the Zendesk API and allows you to export data from Zendesk in Python, update tickets, and trigger automated macros. You can install it by entering !pip3 install zenpy in a code cell in your notebook.

!pip3 install zenpy
from zenpy import Zenpy
import pandas as pd

Connect to the Zendesk API using your API token

Next, you’ll need to login to your Zendesk account and create an API key with which to authenticate. This will require three values: the email address of your Zendesk account, the name of your Zendesk subdomain, and the API token hash. I’ve obfuscated these in the example below. Create a dictionary called credentials in which to store your details.

Pass your credentials dictionary to the Zenpy() class and save the authenticated object as zenpy_client. We’ll need to append our functions to this to allow us to query or update your Zendesk data without hitting authentication issues.

credentials = {
    "email": "REMOVED",
    "subdomain": "REMOVED",
    "token": "REMOVED"
}

zenpy_client = Zenpy(**credentials)

Export open Zendesk tickets

The Zenpy Python package comes with a range of different functions that allow you to search or query the data in your Zendesk account. For this project, we want to export all our open tickets so we can put them in a Pandas dataframe and analyse them.

To fetch your open tickets from Zendesk, you append the search() function to the zenpy_client object, and pass in ticket to the type argument and open to the status argument. This will return our open tickets, which we’ve assigned to open_tickets.

open_tickets = zenpy_client.search(type='ticket', status='open')

To allow us to manipulate the tickets returned by Zenpy, we can use a for loop. We’ll create a list called tickets, then use a for loop to loop over each of the tickets in open_tickets. For each ticket we find, we’ll use the to_dict() function to convert the ticket to a dictionary, and we’ll use append() to add each one to our tickets list.

tickets = []

for ticket in open_tickets:
    tickets.append(ticket.to_dict())

We can look at one of the tickets in the tickets list by passing in its index value. This shows us that we’re dealing with quite a complex nested dictionary, so passing the whole thing into a Pandas dataframe is going to be difficult. Instead, we’ll identify the dictionary values we want to extract and add just these.

tickets[0]
{'assignee_id': REMOVED,
 'brand_id': REMOVED,
 'collaborator_ids': [],
 'created_at': '2021-06-13T00:15:44Z',
 'custom_fields': [{'id': REMOVED, 'value': False},
  {'id': REMOVED, 'value': None}],
 'description': 'REMOVED',
 'due_at': None,
 'external_id': None,
 'fields': [{'id': REMOVED, 'value': False},
  {'id': REMOVED, 'value': None}],
 'forum_topic_id': None,
 'group_id': REMOVED,
 'has_incidents': False,
 'id': REMOVED,
 'organization_id': None,
 'priority': 'urgent',
 'problem_id': None,
 'raw_subject': 'Order queries: I would like to cancel my order',
 'recipient': 'REMOVED',
 'requester_id': REMOVED,
 'sharing_agreement_ids': [],
 'status': 'open',
 'subject': 'Order queries: I would like to cancel my order',
 'submitter_id': REMOVED,
 'tags': ['matt_bot_tag'],
 'type': None,
 'updated_at': '2021-06-13T08:55:32Z',
 'url': 'REMOVED',
 'follower_ids': [],
 'email_cc_ids': [],
 'is_public': True,
 'followup_ids': [],
 'allow_channelback': False,
 'allow_attachments': True,
 'satisfaction_rating': {'assignee_id': None,
  'created_at': None,
  'group_id': None,
  'id': None,
  'requester_id': None,
  'score': 'unoffered',
  'ticket_id': None,
  'updated_at': None,
  'url': None},
 'via': {'channel': 'email',
  'source': {'from_': {'address': 'REMOVED', 'name': 'REMOVED'},
   'rel': None,
   'to': {'name': 'REMOVED', 'address': 'REMOVED'}}}}

Add Zendesk tickets to a Pandas dataframe

To analyse our Zendesk tickets in Pandas we only need a small selection of the numerous columns present in the dictionary. First, we’ll create an empty Pandas dataframe in which to store the data we extract from the dictionaries in the tickets list. We can do this using pd.DataFrame() and pass in a list of column names to assign.

df_tickets = pd.DataFrame(columns=['id', 'requester_id', 'submitter_id', 
                                   'created_at', 'subject', 'description', 
                                   'tags', 'channel', 'status'])

Next we’ll simply use a for loop again to go through each of the dictionaries in our tickets list and extract the values we want using Python dictionary notation. Since we’re putting each ticket’s data on a single row in our dataframe, I’ve called the new dictionary row and have then used the append() function to pass in the row dictionary, along with the important ignore_index=True argument, that allows them to be added.

for ticket in tickets:
    row = {
        'id': ticket['id'],
        'requester_id': ticket['requester_id'],
        'submitter_id': ticket['submitter_id'],
        'created_at': ticket['created_at'],
        'subject': ticket['subject'],
        'description': ticket['description'],        
        'tags': ticket['tags'],
        'channel': ticket['via']['channel'],
        'status': ticket['status'],
    }
    
    df_tickets = df_tickets.append(row, ignore_index=True)    

Finally, we can write the Pandas dataframe of open Zendesk tickets to a CSV file, so we can use it in our next task using the to_csv() function. The index=False value prevents an additional redundant index being added by Pandas.

df_tickets.to_csv('open_tickets.csv', index=False)
df_tickets.head(1)
id requester_id submitter_id created_at subject description tags channel status
0 REMOVED REMOVED REMOVED 2021-06-13T00:15:44Z Order queries: I would like to cancel my order... REMOVED [matt_bot_tag] email open

Now the data are in Pandas, you can analyse them as you wish, create classification models to tag the tickets or their priority, and even use Zenpy to add tags, or trigger macros that cut down on the manual labour your customer service team needs to undertake.

Matt Clarke, Sunday, June 13, 2021

Matt Clarke Matt is an Ecommerce and Marketing Director who uses data science to help in his work. Matt has a Master's degree in Internet Retailing (plus two other Master's degrees in different fields) and specialises in the technical side of ecommerce and marketing.