Introduction

Most of the techniques we've seen in this course have been for numerical features.

The technique we'll look at in this lesson, target encoding,

is instead meant for categorical features.

It's a method of encoding categories as numbers,

like one-hot or label encoding,

with the difference that it also uses the target to create the encoding.

This makes it what we call a supervised feature engineering technique.

​

Target Encoding

Target も参照して

Categoryを定める

Target Encoding

A target encoding is any kind of encoding

that replaces a feature's categories with some number derived from the target.

​

A simple and effective version is

to apply a group aggregation from Lesson 3, like the mean.

Using the Automobiles dataset,

this computes the average price of each vehicle's make:

​

aggregate information across multiple rows

grouped by some category.　　州ごととか

メーカーごとに

平均価格の特徴の列を

追加した

This kind of target encoding is sometimes called a mean平均 encoding.

Applied to a binary target, it‘s also called bin counting. (Other names you might come across include: likelihood encoding, impact encoding, and leave-one-out encoding.)　目的変数のグループの平均の特徴量

Smoothing

An encoding like this presents a couple of problems, however.

​

First are unknown categories. Target encodings create a special risk of overfitting,

which means they need to be trained on an independent "encoding" split.

When you join the encoding to future splits,

Pandas will fill in missing values for any categories not present in the encoding split.

These missing values you would have to impute代入する somehow.

​

Second are rare categories. When a category only occurs a few times in the dataset,

any statistics calculated on its group are unlikely to be very accurate.

In the Automobiles dataset, the mercurcy make only occurs once.

The "mean" price we calculated is just the price of that one vehicle,

which might not be very representative of any Mercuries we might see in the future.

Target encoding rare categories can make overfitting more likely.

A solution to these problems is to add smoothing.

The idea is to blend the in-category average with the overall average. Rare categories get less weight on their category average,

while missing categories just get the overall average.

where weight is a value between 0 and 1

calculated from the category frequency.

An easy way to determine the value for weight is

to compute an m-estimate:

​

An easy way to determine the value for weight is to compute an m-estimate:

where n is the total number of times that category occurs in the data. The parameter m determines the "smoothing factor".

Larger values of m put more weight on the overall estimate.

When choosing a value for m, consider how noisy you expect the categories to be.

Does the price of a vehicle vary a great deal within each make? Would you need a lot of data to get good estimates?

If so, it could be better to choose a larger value for m;

if the average price for each make were relatively stable, a smaller value could be okay.

Use Cases for Target Encoding

Target encoding is great for:

​

High-cardinality features: A feature with a large number of categories can be troublesome to encode: a one-hot encoding would generate too many features and alternatives, like a label encoding, might not be appropriate for that feature.

A target encoding derives numbers for the categories using the feature's most important property: its relationship with the target.

​

Domain-motivated features: From prior experience, you might suspect that a categorical feature should be important even if it scored poorly with a feature metric.

A target encoding can help reveal a feature's true informativeness.

Example - MovieLens1M

​

The MovieLens1M dataset contains one-million movie ratings

by users of the MovieLens website,

with features describing each user and movie.

With over 3000 categories, the Zipcode feature makes a good candidate for target encoding,

and the size of this dataset (over one-million rows) means

we can spare some data to create the encoding.

We'll start by creating a 25% split to train the target encoder.

郵便番号のtarget encoding の列を

追加した

Let's compare the encoded values to the target �to see how informative our encoding might be.

郵便番号と

Ratingに

関係ありかも

The distribution of the encoded Zipcode feature roughly follows the distribution of the actual ratings,

meaning that movie-watchers differed enough in their ratings from zipcode to zipcode that our target encoding was able to capture useful information.