Outlier detection with the interquartile range (IQR)

Before modeling, it's worth checking how "unusual" a variable's extreme values really are. A simple and robust method (it doesn't assume normality) is the interquartile range.

The method

With $Q_1$ and $Q_3$ as the first and third quartiles, the interquartile range is:

IQR = Q_3 - Q_1

A value $x$ is considered an outlier if it falls outside:

[\,Q_1 - 1.5 \cdot IQR,\ \ Q_3 + 1.5 \cdot IQR\,]

Implementation

python

def find_outliers_iqr(series):
    q1, q3 = series.quantile(0.25), series.quantile(0.75)
    iqr = q3 - q1
    lower, upper = q1 - 1.5 * iqr, q3 + 1.5 * iqr
    return series[(series < lower) | (series > upper)]

When not to use it

The 1.5 is a convention, not a law of physics. With highly skewed distributions (income, waiting times) it's better to check visually with a boxplot before discarding data automatically — a genuine outlier may be the most important finding of the analysis.