How to use Pandas to_numeric() to convert strings to numbers

Many Pandas functions require data to be stored in the correct data type, or dtype as it’s known. For example, “£32,232.92” will be recognised as an object data type because...

How to use the Pandas set_index() and reset_index() functions

While many Pandas operations don’t require or benefit from an explicitly named index on the dataframe, named indexes (or indices) can be beneficial for some tasks because a wide range...

How to use lambda functions in Pandas

Lamdba functions are small anonymous functions that don’t need to be defined with a name. If you’re creating a function to solve a specific problem in Pandas and there’s little...

How to measure and reduce Pandas memory usage

While Pandas handles large datasets rather well, it can sometimes struggle with memory in certain situations. Thankfully, there are a few things you can do to reduce the amount of...

How to calculate percentage change between columns in Pandas

When working with Pandas dataframes you’ll often need to calculate the percentage change or percentage difference between the values in two columns. There are various ways to do this in...

How to get a list of national holiday dates in Python

When working with ecommerce and marketing data in time series analysis projects, the dates of national holidays, or bank holidays, can make a big difference to customer behaviour so are...

How to use Spacy EntityRuler for custom Named Entity Recognition

Spacy’s EntityRuler component is one of several rule-based matcher components that can be used to extend the core functionality of the package. It’s really useful for the creation of custom...

How to calculate Spearman's rank correlation coefficient in Pandas

Spearman’s rank correlation coefficient, sometimes called Spearman’s rho, is a nonparametric statistic used to measure rank correlation, or the statistical dependence between the rankings of two variables. It explains how...

How to do custom Named Entity Recognition in Pandas using Spacy

As I showed in my previous tutorial on named entity recognition in Spacy, the EntityRuler allows you to customise Spacy’s default NER model to allow you to create your own...