Data Science

Articles and tutorials on data science using Python

How to create a Google rank checker tool using Python

Import the packages First, open a Jupyter notebook and install my EcommerceTools Python package. EcommerceTools includes a range of modules for performing a range of ecommerce and marketing tasks, including...

How to use the Feefo API for ecommerce competitor analysis

Most ecommerce websites use review platforms, such as Feefo, Trustpilot, and Google Reviews, to allow customers to give feedback on their service and the products they sell. The reviews help...

How to compare time periods using the Google Search Console API

One common task you’ll perform in Google Search Console is to compare the data from two different time periods to see how impressions, clicks, click-through rate (CTR), or average position...

A quick guide to the RFM model for data scientists

The RFM model is probably one of the best known and most widely used customer segmentation models by data driven marketers. It’s used for both measuring customer value and predicting...

How to run time-based SEO tests using Python

One of the problems with search engine optimisation or SEO is that search engine algorithms are essentially black boxes. They analyse so many on-page and off-page factors, and use multiple...

How to create content recommendations using TF IDF

After work, when I’m not learning about data science, practising data science, or writing about data science, I like to browse classic car auction sites looking for cars I can’t...

A quick guide to customer segmentation for B2B e-commerce

Customer segmentation, and the similar and related field of market segmentation, are particularly relevant to the field of business-to-business (B2B) e-commerce. B2B customers often have a higher Customer Lifetime Value...

How to detect Google Search Console anomalies

There are some great anomaly detection models available for Python, which let you examine complex data for a wide range of different anomaly types. In this project, I’ll show you...

How to identify SEO keyword opportunities with Python

One of the most useful Python SEO projects you can undertake is to identify the top keywords for which each of your site’s pages are ranking for. Sometimes, these keywords...

How to add days and subtract days from dates in Pandas

If you regularly work with time series data, one common thing you’ll need to do is add and subtract days from a date. If you tried doing this by hand,...

How to analyse Google Analytics demographics and interests with GAPandas

The demographics and interests data provided in Google Analytics can be a useful way to understand who is visiting your site or purchasing your products, without the need to perform...

How to identify striking distance keywords with Python

Striking distance keywords are those which appear just off the bottom of the first page of search engine results. Keywords that appear on the first page have the greatest visibility...

A quick guide to lead scoring for B2B e-commerce sites

Lead scoring is a Customer Relationship Management (CRM) process that involves segmenting CRM contacts based on their likelihood to make a purchase. Lead scoring is applied to both existing customers...

How to trigger marketing automations using the Mailchimp API in Python

Mailchimp is one of the most widely used email service providers (or ESPs) in ecommerce and marketing. Since it is popular with those who only need its basic campaign features,...

How to create monthly Google Search Console API reports with EcommerceTools

Google Search Console is a really useful tool for search marketers since it shows what is happening data-wise before organic search visitors reach your website. Google Analytics only shows you...

How to use the Mailchimp Marketing Python API with Pandas

In ecommerce, email marketing remains one of the most effective (and cost-effective) digital marketing techniques, especially when combined with data science techniques. The vast amounts of customer data generated in...

How to use the eBay Finding API with Python

The eBay Finding API gives you direct access to eBay search listings using a simple SDK. This API lets you search or query eBay to fetch specific search listings for...

How to export Zendesk tickets into Pandas using Zenpy

The Zendesk customer service platform is widely used by ecommerce businesses, but its functionality for analysing ticket trends and automatically classifying them is somewhat limited. In many cases, you might...

How to query the Google Search Console API with EcommerceTools

The Google Search Console (GSC) API is a great source of information for those working in SEO, marketing, or ecommerce. It can tell you which of your pages are appearing...

How to read Google Sheets data in Pandas with GSpread

GSpread is a Python package that makes it quick and easy to read and write data from Google Sheets spreadsheets stored in your Google Drive into Python. With a tiny...

How to calculate the Lin Rodnitzky Ratio using GAPandas

The Lin Rodnitzky Ratio is a calculation designed to help search engine marketers assess the management of paid search campaigns and account structure. When managing paid search advertising accounts you...

How to analyse product replenishment

Subscription commerce was all the rage for a while, but it’s not really become as popular as many in ecommerce perhaps envisaged. While we may have subscriptions for certain things,...

Data science courses for budding data scientists and data engineers

If you want to change careers and move into the data science or data mining field, as either a data scientist or a data engineer, or simply improve your skills,...

A quick guide to customer segmentation for data scientists

Customer segmentation is the process of using data science techniques to create discrete groups of customers which share common characteristics or attributes. For example, a company might segment customers into...

How to read an RSS feed in Python

RSS feeds have been a mainstay on the web for over 20 years now. These XML-based documents are generated by web servers and designed to be read in RSS feed...

19 Python SEO projects that will improve your site

Although I have never really considered myself a technical SEO, I do need to do quite a bit of SEO work as part of my role as an Ecommerce Director....

How to identify internal and external links using Python

Internal linking helps improve the user experience by recommending related content to users, which both reduces bounce rate, and helps search engine optimisation efforts. While there are no hard and...

How to scrape Google results in three lines of Python code

EcommerceTools makes it really quick and easy to scrape Google search engine results in Python. In this simple project, we’ll use EcommerceTools to search Google for your chosen keywords, scrape...

How to create a simple product recommender system in Pandas

Product recommender systems, or recommendation systems, as they’re also known are ubiquitous on e-commerce websites these days. They’re relatively simple to create and even fairly basic ones can give striking...

15 ways you can use data science to boost ecommerce performance

Major internet retailers, like Walmart and Amazon, have been at the forefront of ecommerce data science and data analytics for many years, contributing lots of interesting papers to data science...

How to create PDF reports in Python using Pandas and Gilfoyle

While reporting is often quite a useful way to stay on top of your data, it’s also something you can automate to save time, even if your reports include custom...

How to create monthly Google Analytics reports in Pandas

Like most people who work in ecommerce and marketing, I spend a lot of time in Google Analytics. It’s a great tool, but when reporting on the numbers, it helps...

How to segment your customers using EcommerceTools

Customer segmentation can give you huge insights into your business and identify a whole range of different things about your customers, allowing you to change your marketing and improve results....

How to use EcommerceTools for technical SEO

There’s often a lot of faffing around required to get marketing and ecommerce data from various systems into Pandas so you can analyse it, or use it within more complex...

How to segment customers using RFM and ABC

While the Recency, Frequency, Monetary value or RFM approach to customer segmentation might be old, it’s based on sound science, so no matter what customer model you’re building, it’s generally...

How to perform a customer cohort analysis in Pandas

Cohort analysis is unlike most other customer segmentation techniques in that it typically uses a time-based element. It’s typically used to segment customers into groups, or cohorts, based on their...

How to machine translate product descriptions

Whether you’re analysing content written in other languages using Natural Language Processing, or you want to assist your content team by translating their writing into other languages, machine translating software...

How to identify near duplicate content using LMS

In those ecommerce businesses where relatively few products are launched and products have a relatively long lifecycle, copywriters tend to be targeted on producing unique content that sells the benefits...

How to create a product and price metadata scraper

In ecommerce, price monitoring is a really important consideration. If you offer your products at a price which is too high within the market, you may lose sales to rivals,...

How to calculate CLV using BG/NBD and Gamma-Gamma

Calculating Customer Lifetime Value or CLV is considered a really important thing in marketing and ecommerce, yet most companies can’t do it properly. This clever metric tells you the predicted...

How to assign RFM scores with quantile-based discretization

RFM segmentation is one of the oldest and most effective ways to segment customers. [RFM models] (/data-science/a-quick-guide-to-the-rfm-model) are based on three simple values - recency, frequency, and monetary value -...

How to analyse product consumption and repurchase rates

You can learn many things about your products from the purchase behaviours and product consumption and product replenishment of your customers. Some items are purchased individually, some items are purchased...

How to use Spintax to create content and ad copy in Python

If you’ve never heard of Spintax, you’ll definitely be aware of its typical usages. This text string replacement “language” is most commonly used for text spinning or article spinning, but...

How to scrape schema.org metadata using Python

As I’ve mentioned in previous posts on web scraping, the most efficient way to scrape data is to identify what Schema.org metadata is in use and then create a microdata...

How to scrape People Also Ask data using Python

People Also Ask or PAA boxes have been becoming increasingly common in Google’s search results over the past few years. They show a range of questions and answers related to...

How to scrape Google search results using Python

Although I suspect you are probably not technically allowed to do it, I doubt there’s an SEO in the land who hasn’t scraped Google search engine results to analyse them,...

How to join Google Analytics and Google Search Console data

Neither Google Search Console nor Google Analytics gives you access to the data found in both systems in one place. However, with a bit of ingenuity and some relatively simple...

How to identify SEO keywords using Google Autocomplete

The Google Autocomplete feature, or Google Suggest as it was previously known, has become a part of everyday life for us all. Start typing a search term into Google, and...

How to engineer customer purchase latency features

Purchase latency or customer latency is a measure of the number of days between a customer’s orders and is one of the most powerful features in many propensity and churn...

How to create targeted B2B company sector datasets

As I explained in my previous post, many B2B ecommerce businesses spend huge amounts on procuring third-party data for companies they wish to target. However, with some data science skills...

How to create a UK data science jobs dataset

According to the Harvard Business Review, the role of data scientist is said to be “the sexiest job of the 21st century” . Data science and data engineering skills are...

How to create a dataset containing all UK companies

In B2B ecommerce, there are two main approaches to new customer acquisition: you either rely on your website to acquire customers for you, or you target specific customers through sales...

How to count indexed pages using Python

One quick and easy way to understand the size of a website, and its growth rate, is to examine the number of its web pages Google has indexed. You can...

How to calculate safety stock and reorder point

Although the techniques for reducing its impact have existed for decades, inventory management is still a huge issue in many businesses. Various things happen that can result in costly stock...

How to calculate operations management metrics in Python

Successful operations management is crucial to the overall growth of an ecommerce business. While those in ecommerce, marketing, or data science, can work together the sales coming in and encourage...

How to calculate marketing metrics in Python

Marketers can be just as obsessive about data as data scientists, so there are an abundance of well-researched marketing metrics available for analysing marketing performance. Most of the commonly used...

How to calculate customer experience metrics in Python

Customers are expensive to acquire but generate more and more profit as time goes on. Providing you nurture them, treat them kindly, and apologise and fix any mistakes that occur,...

How to calculate category management metrics in Python

Category management is a retail technique that breaks down a company’s product range into groups of related items, such as categories, or subcategories, or by their product type. By running...

How to access the Google Knowledge Graph Search API

The Google Knowledge Graph database includes an astronomical amount of data on almost every topic you can think of, allowing Google to create Knowledge Panels and infoboxes that summarise search...

A quick guide to catalogue marketing data science

Catalogue marketing is dying out. Over the past few years, virtually all the UK’s top catalogue retailers have stopped printing on paper and successfully transitioned their businesses online, either to...

How to use Extruct to identify Schema.org metadata usage

The downside to building datasets using web scraping is that every site has custom HTML. If you scrape sites in this way, you’ll forever be building bespoke scrapers, and they’ll...

How to unzip files with Python

Most very large datasets tend to get compressed on servers to preserve storage space and bandwidth and allow them to be downloaded more quickly by end users. Python includes some...

How to unserialize serialized PHP arrays using Python

If you regularly work with ecommerce data, you’re likely to have encountered PHP serialized arrays or objects. Serialization is a process used to take a complex data structure, such as...

How to send data to Google Analytics in Python with PyGAMP

The Google Analytics Measurement Protocol API lets you add data to your GA account that hasn’t been triggered by a user visiting a web page. Since it’s so flexible, you...

How to scrape Open Graph protocol data using Python

Many websites include Open Graph protocol data in their document head. This structured data allows social networks, such as Facebook and Twitter, to access specific elements of the page’s content...

How to scrape and parse a robots.txt file using Python

When scraping websites, and when checking how well a site is configured for crawling, it pays to carefully check and parse the site’s robots.txt file. This file, which should be...

How to scrape a site's page titles and meta descriptions

Scraping the titles and meta descriptions from every page on a site can tell you a great deal about its content, the underlying content strategy, or product ranges, and many...

How to scan a site for 404 errors and 301 redirect chains

Both 404 page not found errors and 301 redirect chains can be costly and damaging to the performance of a website. They’re both easy to introduce, especially on ecommerce sites...

How to resize and compress images with TinyPNG

Large, uncompressed images slow down your site, increase bandwidth costs, harm the user experience, and impact search engine rankings. In this project, I’ll show you how you can bulk resize...

How to parse XML sitemaps using Python

XML sitemaps are designed to make life easier for search engines by providing an index of a site’s URLs. However, they’re also a useful tool in competitor analysis and allow...

How to parse URL structures using Python

URLs often contain useful information that can be used to analyse a website, a user’s search, or the breakdown of content present in each section. While they often look pretty...

How to identify keyword cannibalisation using Python

Keyword cannibalisation occurs when you have several pages ranking for the same phrase, effectively putting them in competition with each other for search engine rankings. Since Google generally only shows...

How to download files with Python

In many data science projects you may need to download remote data, such as images, CSV files, or compressed data. Python makes it fairly straightforward to download files within your...

How to calculate Economic Order Quantity in Python

The Economic Order Quantity or EOQ represents the optimum purchase quantity for a given product, while aiming to minimise holding costs, shortage costs, and order costs. It’s most commonly calculated...

How to build a web scraper using Requests-HTML

Unless you’re building a large and complex web scraper using Scrapy or Selenium, it’s probable that you’ll utilise Requests and Beautiful Soup. These two packages are brilliant for web scraping....

How to audit a site's Core Web Vitals using Python

Back in 2020, Google introduced Web Vitals, a set of metrics is designed to help site owners to optimise the user experience on their websites, so pages are quick to...

How to analyse Pandas dataframes using SQL with PandaSQL

If, like me, you’ve come from a background where you made heavy use of SQL, then getting to grips with filtering, subsetting, and selecting data in Pandas can be a...

How to analyse non-ranking pages and search index bloat

If your site’s pages aren’t indexed by Google, you’re obviously not going to generate any traffic to them, so you’ll want to check that everything you expect to be present...

How to access the Google Search Console API using Python

Google Search Console contains loads of really useful information for technical SEO. However, there are limits to what you can do using the front-end interface, and it takes time to...

How to use Screaming Frog from the command line

The Screaming Frog SEO Spider Tool is widely used in digital marketing and ecommerce. It provides a user-friendly interface to a powerful site crawler and scraper that can be used...

How to send a Slack message in Python using webhooks

Slack is a great tool for data scientists and data engineers and is now being adopted across businesses, so it’s probable that you already use it in your workplace. Besides...

How to geocode and map addresses using GeoPy

In the field sales sector, one common thing you’ll want to do is identify all the potential clients you have within a particular region, so you can assign your team...

How to create paid search keywords using Pandas

Setting up keywords for new paid search accounts can be a repetitive and time-consuming process. While it’s historically been done using Excel, many digital marketers are now taking advantage of...

How to create a Python web scraper using Beautiful Soup

Web scraping is a really useful skill in data science. We obviously need data for our models and analyses, but it’s not always easily available, so building our own datasets...

How to write better code using DRY and Do One Thing

DRY, or Don’t Repeat Yourself, and the “Do One Thing” methodology are designed to help software engineers and data scientists create better functions. Code that isn’t written using DRY tends...

How to visualise data with quirky hand-drawn plots

Charts and plots can often look a bit stale and professional, which might not be appropriate in every setting. If you want to dumb-down your charts and make them look...

How to visualise conversion funnels with Plotly

Funnels are arguably one of the most powerful data visualisations you can use within the ecommerce field. At a glance, they can show you the proportion of customers entering at...

How to use style guidelines to improve your Python code

The flexibility of programming languages like Python means that any code you write to tackle a given problem will differ in approach and style to code written by someone else....

How to use SQLite in Python

SQLite is a relational database management system (RDBMS) that is easy to access within Python and other languages. Unlike MySQL, PostgreSQL, and other databases, SQLite uses a serverless design, so...

How to use operators in Python

For data scientists, Python operators are one of the most powerful and widely used features of this language. These special symbols or characters tell Python to perform some sort of...

How to use lists in Python

Lists are one of the most widely used data storage objects or data types within Python and are used throughout every data science package. Along with the dictionary, tuple, and...

How to use Git for your data science projects

Git is the world’s most widely used version control system and is an essential tool for data scientists, especially those collaborating on projects with others. You’ll need to be able...

How to use docstrings to improve your Python code

Docstrings are comment blocks that are added to the top of Python functions to explain the purpose of the function, describe the arguments that it accepts, and explain what the...

How to use the Pandas value_counts() function

The Pandas value_counts() function can be used to count the number of times a value occurs within a dataframe column or series, as well as calculating frequency distributions. Here’s a...

How to query MySQL and other databases using Pandas

For years, I used to spend much of my time performing Exploratory Data Analysis directly in SQL. Over time, the queries I wrote became very complicated, and it was often...

How to open, read, and write to files in Python

While data scientists may do nearly everything in Pandas, we also need to perform file operations in regular Python and in applications not tied to dataframes. Thankfully, Python makes it...

The four Python data science libraries you need to learn

There are hundreds of excellent Python data science libraries and packages that you’ll encounter when working on data science projects. However, there are four of them that you’ll probably use...

How to visualise text data using word clouds in Python

Word clouds (also known as tag clouds, wordles, or weighted lists) have been around since the mid nineties and are one of the most effective data visualisations for representing the...

How to visualise statistical distributions with Seaborn

One of the key steps in the Exploratory Data Analysis process that comes before model development is to understand the statistical distribution of the variables or features within the data...

How to visualise data using Venn diagrams in Matplotlib

The Venn diagram is one of the most intuitive data visualisations for showing the overlap between two or three groups, or “sets”, of data. These diagrams were created in the...

How to visualise data using line charts in Seaborn

Line charts, line graphs, or line plots are among the most widely used data visualisations. They’re ideal for time series data in which you’re plot a metric on the y...

How to visualise data using barplots in Seaborn

Barplots or bar charts are probably the most widely used visualisation for displaying and comparing categorical variables. They’re very easy to understand and are quick and easy to generate.

How to visualise correlations using Pandas and Seaborn

Pearson’s product-moment correlation, or Pearson’s R, is a statistical method commonly used in data science to measure the strength of the linear relationship between variables. If you can identify existing...

How to visualise categorical data in Seaborn

Categorical data can be visualised in many ways, and there’s no requirement to stick to the standard bar chart. Here are a selection of attractive Seaborn charts, graphs, and plots...

How to install the NVIDIA Data Science Stack on Ubuntu 20.04

One of the most annoying aspects of working with GPU-accelerated data science software, such as NVIDIA Rapids, TensorFlow, PyTorch and XGBoost, is that it can sometimes be very complicated and...

How to create desktop data science apps using Nativefier

There are numerous websites I use for my work that don’t have dedicated desktop applications designed for Ubuntu Linux, such as GitHub, GitHub Gists, GitLab and Jira. However, it’s now...

How to create an Ubuntu desktop entry to run Jupyter

Despite the massive improvements to usability on Linux over the years, it still remains unnecessarily complicated to create shortcut icons on Gnome.

How to build a data science workstation

If you’re working in data science, and especially if you’re working in deep learning, you’re going to need a decent workstation in order to be productive. Earlier this year I...

How to visualise analytics data using heatmaps in Seaborn

Heatmaps are one of the most intuitive ways to display data across two dimensions, and they work particularly well on temporal data, such as web analytics metrics. They’re a great...

How to visualise RFM data using treemaps

Recent papers on the Recency, Frequency, Monetary (RFM) model, such as this one by Inanc Kabasakal which was published earlier this year, have started to adopt text-based labels to help...

How to visualise data using scatterplots in Seaborn

Scatterplots, scatter graphs, scatter charts, or scattergrams, are one of the most popular mathematical plots and represent one of the best ways to visualise the relationship of data on two...

How to visualise data using histograms in Pandas

During the Exploratory Data Analysis or EDA stage one of the key things you’ll want to do is understand the statistical distribution of your data. Histograms are one of the...

How to visualise data using boxplots in Seaborn

The boxplot, or box-and-whisker diagram, is one of the most useful ways to visualise statistical distributions in data. While they can seem a bit unintuitive when you first look at...

How to select, filter, and subset data in Pandas dataframes

Selecting, filtering and subsetting data is probably the most common task you’ll undertake if you work with data. It allows you to extract subsets of data where row or column...

How to resample time series data in Pandas

When working with time series data, such as web analytics data or ecommerce sales, the time series format in your dataset might not be ideal for the analysis you’re performing...

How to reformat dates in Pandas

If you regularly work with time series data in Pandas it’s probable that you’ll sometimes need to convert dates or datetimes and extract additional features from them.

How to import data into Pandas dataframes

Pandas allows you to import data from a wide range of data sources directly into a dataframe. These can be static files, such as CSV, TSV, fixed width files, Microsoft...

How to group and aggregate transactional data using Pandas

Transactional item data can be used to create a number of other useful datasets to help you analyse ecommerce products and customers. From the core list of items purchased you...

How to analyse search traffic using the Google Trends API

The things we search for online can reveal a remarkable amount about us, even when viewed in aggregate on an anonymous level. For many years, Google has made some of...

How to use identify visually similar images using hashing

Image hashing (or image fingerprinting) is a technique that is used to convert an image to an alphanumeric string. While this might sound somewhat pointless, it actually has a number...

How to create an ABC XYZ inventory classification model

As everyone who works in ecommerce will know, stock-outs on your key lines can have a massive negative impact on sales and your marketing costs. In many cases, you’ll be...

How to create an ABC inventory classification model

ABC inventory classification has been one of the most widely used methods of stock control in operations management for decades. It’s an intentionally simple system in which products are assigned...

How to connect to MySQL via an SSH tunnel in Python

Many MySQL databases are configured to accept connections from other servers on the local network and will reject connections from remote machines. Ordinarily, you could work around this by creating...

How to calculate relative dates for Google Analytics queries

The Google Analytics add-on for Google Sheets allows you to use the Google Analytics reporting API to create custom weekly reports and schedule them to run. However, to run a...

How to create an ecommerce trading calendar using Pandas

In both B2C and B2B ecommerce, special trading periods such as Christmas, Mothers’ Day, and Valentines’ Day can often greatly contribute to sales. Indeed, the introduction of Black Friday sales...

Dell Precision 7750 mobile data science workstation review

The Precision 7000 series is the top of the range mobile workstation laptop from Dell and is aimed firmly at professional users who are doing high-end GPU-accelerated work, whether that’s...

How to use the Pandas melt function to reshape wide format data

When you gain access to a new dataset, chances are, it’s probably not in the format you require for analysis or modeling. The most common problem you’ll encounter is datasets...

How to use the Apriori algorithm for Market Basket Analysis

Market Basket Analysis, or MBA, is a subset of affinity analysis and has been used in the retail sector for many years. It provides a computational method for identifying common...

How to scrape JSON-LD competitor reviews using Extruct

In the ecommerce sector, you can learn a lot about your competitors and the expectations of your customers by analysing the reviews their customers leave for products and service on...

How to scrape competitor technology data in Python

In ecommerce, it pays to watch what your competitors are doing, so over the past decade or so in which I’ve managed ecommerce businesses, I’ve regularly undertaken competitor analyses. They’re...

How to create a Pandas dataframe

The massive versatility of Pandas means that you can create dataframes from almost any type of raw data. Whether you have a list, a list of lists, a dictionary, a...

How to create a collaborative filtering recommender system

Recommender systems, or recommendation engines as they’re also known, are everywhere these days. Whether you’re looking for books on Amazon, tracks on Spotify, movies on Netflix or a date on...

How to use GAPandas to view your Google Analytics data

Over the past decade I’ve written more Google Analytics API queries than I can remember. Initially, I favoured PHP for these (and still do for permanent web-based applications utilising GA...

How to use scikit-learn datasets in data science projects

The scikit-learn package comes with a range of small built-in toy datasets that are ideal for using in test projects and applications. As they’re part of the scikit-learn package, you...

How to use Python regular expressions to extract information

Regular expressions are used for pattern matching in programming, allowing you to identify or extract very specific pieces of text from a string or document. They’re very powerful and extremely...

How to engineer date features using Pandas

When dealing with temporal or time series data, the dates themselves often yield information that can vastly improve the performance of your model. However, to get the best from these...

How to create a Python virtual environment for Jupyter

When you use pip to install Python packages from The Python Package Index (PyPi) they get stored in your site-packages directory and are used across your system whenever you run...