How to trigger marketing automations using the Mailchimp API in Python

Picture by Phil Goodwin, Unsplash.

21 minutes to read

Mailchimp is one of the most widely used email service providers (or ESPs) in ecommerce and marketing. Since it is popular with those who only need its basic campaign features, it’s built-in functionality for creating marketing automation emails can sometimes be overlooked as being too basic, compared to more sophisticated platforms.

While Mailchimp’s marketing automation functionality isn’t currently up there with the best, it’s actually pretty good for basic needs, and can be quite powerful if you can make use of the Mailchimp marketing API. With the API, and some data science skills, you can actually create some quite impressive marketing automation email campaigns.

Practical applications in ecommerce

For example, let’s say you sell washing machines and find that customers who purchase them always get confused about how to install them, leading to lots of customer service tickets or emails. To resolve this, you could run a database query to identify new washing purchasers, add a tag called purchased_washing_machine, and trigger an automation email in Mailchimp to give them some tips and advice.

Since these emails are timely and relevant, open rates and click through rates tend to be very strong. They’re also a great way to cross-sell other items that customers might want, and can be used to trigger replenishment emails later down the product lifecycle.

In this project, I’ll show how you can use the Mailchimp marketing API to create the tags you need to set marketing automation triggers in Mailchimp, so you can create your own marketing automation email campaigns.

Triggering marketing automation emails

While Mailchimp is good for creating basic marketing automation campaigns, it’s not as feature rich as some other systems, especially if you don’t have all your ecommerce data in the system, or you want to create marketing automation tags to trigger emails using data from models or external data sources.

However, if you’re happy to put in the extra work, it’s a great budget solution for those of a technical nature who can exploit the API to make it do more than the average marketer could achieve. The frustrating thing is that its internal rules engine for creating segments is somewhat limited, particularly in the standard version.

For example, you can’t create complex rules, and you’re limited to the number of fields you can use to create the triggers that cause emails to be sent automatically. You need to plan for this to get the best out of the platform.

Prior to developing your marketing automation application, I’d recommend manually creating some tags using the Mailchimp backend and then checking to ensure you can create an automation that does what you want. There are some annoying shortcomings in the rules engine that mean only certain scenarios are possible, but pre-development prototyping can find these and save wasted time.

Ecommerce Picture by Negative Space, Pexels.

Understanding Mailchimp’s data structure

To understand what data you can put into Mailchimp, we first need to go over Mailchimp’s underlying data structure. This comprises three simple features: merge fields, tags, and segments:

Merge fields

Merge fields were formerly known as merge vars and exist as columns of data within the Mailchimp audience designed to hold variable data. The merge field name is fixed, but the value could be different across customers. For example, you might have a merge field called TOTAL_ORDERS which stores the total number of orders each customer has placed.

Merge fields are specific to a Mailchimp list. You can add these manually using the Mailchimp admin panel, or you can create them via the API using the client.lists.add_list_merge_field(). They can be required for each user and you can control whether users can see them via your Mailchimp preferences centre, or whether they’re hidden from subscribers.

Merge fields can have a range of different “types” which alter the way the data are stored and how the field appears in the backend. These include: text, number, address, phone, date, url, imageurl, radio, dropdown, birthday, and zip.

The API can be used to list all the merge fields, add merge fields, get a specific merge field’s settings, update a merge field, or delete a merge field. When adding list members, you can easily update the values you push to the API to populate these fields.

Segments

Segments are custom-made groups of email subscribers whose merge field or tag data matches a defined set of criteria. The usual approach to using Mailchimp segments is to create and populate the appropriate merge fields and tags regularly, and then use segments to create specific subsets of subscribers to target.

For example, you might create a segment contain all subscribers with the merge field of customer, and who have spent over 100 per order in the past year, and have been tagged with the purchased_washing_machine tag.

You can create complex segments using up to five features using the basic version of Mailchimp, but this is more feature-rich in Mailchimp Pro. The downside is that the rules aren’t that flexible, and I find this is where problems usually arise.

Gmail Picture by Krsto Jevtic, Unsplash.

Load the packages

For this project we’ll be using the Mailchimp Marketing API. I’ve covered the basic usage of this API in my previous article, so you may want to check that out before you start. The package is available via PyPi, so install this first and then import the packages below.

!pip3 install mailchimp-marketing

import pandas as pd
import hashlib
import mailchimp_marketing as MailchimpMarketing
from mailchimp_marketing.api_client import ApiClientError

Configure the API connection

Next, we need to extract the API configuration parameters from your Mailchimp account so we can authenticate against Mailchimp and retrieve and update the data in your Mailchimp account.

To connect to the Mailchimp API, you will need three variables that have to be extracted from your Mailchimp account. These are the API key, the server prefix, and the list ID, or audience ID. They’re not particularly easy to find. Here’s how it’s done at present:

API key

Login to Mailchimp.
Click the M icon in the bottom right corner > Profile
Extras > API keys

Server prefix

Login to Mailchimp
Look at the URL in the address bar
Extract the first set of characters, i.e. us2

List ID

Login to Mailchimp
Click Audience > View contacts
Click Settings > Audience name and defaults
Find the Audience ID section and look for a code like this: sd878f7dsf

To save re-entering these throughout the code, we’ll define some constants in which to store them for later use.

API_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-us2'
SERVER_PREFIX = 'us2'
LIST_ID = 'xxxxxxxxx'

Create your tags in Mailchimp

Since we’re going to trigger our marketing automations by creating and assigning tags in Mailchimp, the first thing you’ll need to do is create the tags you want to use and assign them to a test email address, such as your own.

You can do this by finding your email address within the Mailchimp list, and then clicking the plus icon in the tags section to add an existing tag or create a new one and add it to the email subscriber. I’d recommend using a simple naming convention for you tags, such as lowercase letters and underscores only.

Return a list of tags and tag IDs

In order to assign an existing tag to your users you will need the tag_id. However, annoyingly, this isn’t visible within the Mailchimp interface. However, there’s an API method called client.lists.tag_search() that returns a list of all tags on a list.

We’ll create a function called tag_search() to which we can pass the list_id obtained above via a try except block. This will connection to the Mailchimp API using your API key and server prefix and return a dictionary containing all the tags found on that list_id. You can then identify the tag_id for the tag you wish to assign to other subscribers.

def tag_search(list_id):
    """Return all tags on a list"""
    try:
        client = MailchimpMarketing.Client()
        client.set_config({
            "api_key": API_KEY,
            "server": SERVER_PREFIX
        })

        response = client.lists.tag_search(list_id)
        return response
    except ApiClientError as error:
          print("Error: {}".format(error.text))

response = tag_search('xxxxxxxxx')
response

{'tags': [
  {'id': 123, 'name': 'Installed Pied Piper'},
  {'id': 135, 'name': 'Visited Hoolicon'},
  {'id': 244, 'name': 'Shared app'},
  {'id': 148, 'name': 'Heavy app user'}],
 'total_items': 4}

Get member tags

Next, we’ll create another helper function called get_member_tags() to use to check that any updates we make to a list subscriber have worked. We’ll pass in the list_id and the email, connect to the API and then fetch the tags assigned to that user using the get_list_member_tags() function.

Importantly, this requires the email address you pass to be hashed using the MD5 hashing algorithm. We can ensure this is in the right format for Mailchimp by using this code: hashlib.md5(email.encode('utf-8')).hexdigest(). After running that, we get a dictionary containing the tags for our given email address.

def get_member_tags(list_id, email):
    """Get member tags."""
    try:
        client = MailchimpMarketing.Client()
        client.set_config({
            "api_key": API_KEY,
            "server": SERVER_PREFIX
        })

        subscriber_hash = hashlib.md5(email.encode('utf-8')).hexdigest()
        response = client.lists.get_list_member_tags(list_id, subscriber_hash)

        return response
    except ApiClientError as error:
          print("Error: {}".format(error.text))

response = get_member_tags('xxxxxxxxx', 'richard@example.com')
response

{'tags': [{'id': 123,
   'name': 'installed_pied_piper',
   'date_added': '2021-07-13T08:23:06+00:00'}],
 'total_items': 1}

Adding or removing tags

We’re going to trigger our marketing automation by adding a tag to specific email addresses. To do this, we need to find the tag_id using the function we created a moment ago. We can use then use the client.lists.batch_segment_members() API function to add or remove email subscribers to the tag.

Our function takes a list of emails. That list can contain a single email address or up to 500, allowing us to perform a bulk update or bulk remove on a given tag. The function below provides a neat wrapper to it, and lets you add or remove a tag from a list of emails by setting the add parameter to True or False.

emails_list = ['richard@example.com']

def update_tags(list_id, tag_id, emails_list, add=True):
    """Assign or remove a tag from a list of emails within a Mailchimp list. 
    
    Args:
        list_id (string): Mailchimp list ID
        tag_id (int): Mailchimp tag ID
        emails_list (list): List of email addresses in the Mailchimp list to update
        add (boolean, optional, default=True): Set to True to add or update tags, or False to remove tags. 
    
    Returns:
        response: API response or 
    
    """
    try:
        client = MailchimpMarketing.Client()
        client.set_config({
            "api_key": API_KEY,
            "server": SERVER_PREFIX
        })

        if add:
            response = client.lists.batch_segment_members({"members_to_add": emails_list}, list_id, tag_id)
        else:
            response = client.lists.batch_segment_members({"members_to_remove": emails_list}, list_id, tag_id)
        return response
    except ApiClientError as error:
          return format(error.text)

response = update_tags('xxxxxxxxx', '123', emails_list, add=True)
response

{'members_added': [],
 'members_removed': [],
 'errors': [{'email_addresses': ['richard@example.com'],
   'error': 'Email addresses already exist in the static segment'}],
 'total_added': 0,
 'total_removed': 0,
 'error_count': 1,
 '_links': [{'rel': 'self',
   'href': 'https://us2.api.mailchimp.com/3.0/lists/xxxxxxxxx/segments/123',
   'method': 'GET',
   'targetSchema': 'https://us2.api.mailchimp.com/schema/3.0/Definitions/Lists/Segments/Response.json'},
  {'rel': 'parent',
   'href': 'https://us2.api.mailchimp.com/3.0/lists/b5ed477df9/segments',
   'method': 'GET',
   'targetSchema': 'https://us2.api.mailchimp.com/schema/3.0/Definitions/Lists/Segments/CollectionResponse.json',
   'schema': 'https://us2.api.mailchimp.com/schema/3.0/Paths/Lists/Segments/Collection.json'},
  {'rel': 'delete',
   'href': 'https://us2.api.mailchimp.com/3.0/lists/xxxxxxxxx/segments/123',
   'method': 'DELETE'},
  {'rel': 'update',
   'href': 'https://us2.api.mailchimp.com/3.0/lists/xxxxxxxxx/segments/123',
   'method': 'PATCH',
   'targetSchema': 'https://us2.api.mailchimp.com/schema/3.0/Definitions/Lists/Segments/Response.json',
   'schema': 'https://us2.api.mailchimp.com/schema/3.0/Definitions/Lists/Segments/PATCH.json'},
  {'rel': 'members',
   'href': 'https://us2.api.mailchimp.com/3.0/lists/xxxxxxxxx/segments/123/members',
   'method': 'GET',
   'targetSchema': 'https://us2.api.mailchimp.com/schema/3.0/Definitions/Lists/Segments/Members/Response.json'}]}

Bulk updating larger email lists

Since the client.lists.batch_segment_members() function only accepts up to 500 email addresses, you will need to make multiple requests if the volume of emails you wish to tag exceeds this amount. Next we will modify the function above, so it can handle a longer list of emails, batch them up into groups of 500 or fewer, and then run each batch through the API.

First, we’ll create a helper function called create_batches() that will take our list of emails, and our batch size, i.e. 500. It will break the original list up into batches of 500, and put the remainder in a final list. It will then put these lists into another list and return that. A simple for loop can then be used to access each batch.

def create_batches(emails_list, batch_size=500):
    """Splits a list of emails into batches of size n and returns a list of batches.
    
    Args:
        emails_list: List of emails
        batch_size (optional, int): Maximum batch size (default is 500)
    
    Returns:
        batches: List of lists of batches of 500 or fewer emails.
    """
    
    batches = [emails[i:i + batch_size] for i in range(0, len(emails), batch_size)] 
    return batches

If we create a test list of emails and run it through create_batches() with the batch_size argument set to 1, we should get back a single list, containing a list for each batch of one email.

This works fine, so the next step is to incorporate the code into the update_tags() function we created in the previous step, so it can support massive lists of emails and can run them through the Mailchimp API in batches of 500 at a time.

emails = ['richard@example.com', 'erlich@example.com', 'jared@example.com', 'gilfoyle@example.com']
batches = create_batches(emails, batch_size=1)

batches

[['richard@example.com'],
 ['erlich@example.com'],
 ['jared@example.com'],
 ['gilfoyle@example.com']]

Now, we’ll duplicate the update_tags() function and make some modifications so it supports batch processing. We’ll add a function argument for batch_size which we’ll set to the 500 email limit of Mailchimp, then we’ll use create_batches() to take the original list of emails and split it into chunks of 500 or fewer, assigning the list of lists to batches.

Then, rather than just running the batch_segment_members() function on the entire emails_list list, which might hit the 500 email limit, we’ll use a for loop. For each batch of 500 or fewer emails we encounter, we’ll pass the list stored in batch to the batch_segment_members() function.

If we get back a response from the API, we’ll add it to the responses list, otherwise we’ll append() an exception error, and then return the list of responses as the output.

def batch_update_tags(list_id, tag_id, emails_list, add=True, batch_size=500):
    """Bulk tag a list of up to 500 emails. """
    
    responses = []
    
    try:
        client = MailchimpMarketing.Client()
        client.set_config({
            "api_key": API_KEY,
            "server": SERVER_PREFIX
        })

        batches = create_batches(emails_list, batch_size=batch_size)
        
        if add:
            for batch in batches:
                response = client.lists.batch_segment_members({"members_to_add": batch}, list_id, tag_id)
                
                if response:
                    responses.append(response)      
        else:
            for batch in batches:
                response = client.lists.batch_segment_members({"members_to_remove": batch}, list_id, tag_id)
                
                if response:
                    responses.append(response)  

    except ApiClientError as error:
        responses.append(format(error.text))
            
    return responses

Running the function on our list will add the tag if the user exists in the list, or will return an error if they’re not present. If there are more than 500 emails, it will create batches and run each batch through the API, thus bypassing the 500 email limit.

responses = batch_update_tags('xxxxxxxxxxx', '123', emails_list, add=True, batch_size=1)

Putting it into production

The final steps are to put it into production. I’m going to skip this step, as it depends on your specific environment. To get the list of emails, you simply need to write a database query to fetch emails that match your given criteria, i.e. “installed app” and then pass that to batch_update_tags().

You’ll probably want to run these using an orchestration system such as Airflow or Luigi. Check out my guide to creating pipelines with Apache Airflow for some general pointers on how to get this up and running.

Matt Clarke, Friday, July 16, 2021

Matt Clarke Matt is an Ecommerce and Marketing Director who uses data science to help in his work. Matt has a Master's degree in Internet Retailing (plus two other Master's degrees in different fields) and specialises in the technical side of ecommerce and marketing.