1 of 39

2 of 39

Introduction to

Recommender Systems

3 of 39

Scaling Personalized Recommendations

Using Photon ML

4 of 39

Challenges for personalization

  • Personalized recommender system should...
    • Adapt behavior to individual users or items with more data
    • Address the different granularities (country, region, ...) and dimensions (member, job, company, ...) observed in data.

4

5 of 39

Challenges for personalization

  • Personalized recommender system should...
    • Adapt behavior to individual users or items with more data
    • Address the different granularities (country, region, ...) and dimensions (member, job, company, ...) observed in data.

​

  • Good models should .. ?
    • Many possible approaches

​

5

6 of 39

Matrix Factorization (MF)

  • Predicting the response of user i on item j:

​

​

​

  • : User latent factor
  • : Item latent factor
  • : Expectation of response
  • : Link function

7 of 39

Matrix Factorization (MF)

  • Predicting the response of user i on item j:

​

​

​

  • Pro: Excellent in personalization with not-so-sparse data
  • Con: Does not leverage features

8 of 39

Generalized Linear Model (GLM)

  • Predicting the response of user i on item j:

​

​

​

  • : Feature vector
  • : Coefficient vector
  • : Expectation of response
  • : Link function

9 of 39

Generalized Linear Model (GLM)

  • Predicting the response of user i on item j:

​

​

​

  • Pro: Interpretable, explore many features
  • Con: Non-trivial to capture signals from different granularity

10 of 39

Generalized Additive Model (GAM)

  • Predicting the response of user i on item j:

​

​

​

  • : Smooth function of the pth feature
  • : Expectation of response
  • : Link function

11 of 39

Generalized Additive Model (GAM)

  • Predicting the response of user i on item j:

​

​

​

  • Pro: More flexible than GLM
  • Cons: Overfitting, model selection

12 of 39

Factorization Machine (FM)

  • Predicting the response of user i on item j:

​

​

​

  • : Latent feature projection matrix
  • : Expectation of response
  • : Link function

13 of 39

(Deep) Neural Network (DNN)

14 of 39

The Paradox of Choice - More is Less

  • Multiple rounds of experiments required to determine which model to use.
  • Different models usually found in different libraries implemented in different languages and have different input/output settings.
  • Significant effort required to build a new model.

​

14

15 of 39

The Paradox of Choice - More is Less

  • Multiple rounds of experiments required to determine which model to use.
  • Different models usually found in different libraries implemented in different languages and have different input/output settings.
  • Significant effort required to build a new model.

​

  • Need a unified framework to harness them all, experimenting new modeling ideas with high agility.

15

16 of 39

GAME: Generalized Additive Mixed-Effect Model

17 of 39

GAME as a framework, optimizer and library

18 of 39

GAME as a Framework

  • Unifies and mixes different models into a principled additive model.
  • Predicting the response of user i on item j:

​

​

  • : An effect (model)
  • : Expectation of response
  • : Link function

19 of 39

GAME as an Optimizer

  • Predicting the response of user i on item j:

​

​

  • Coordinate Descent / Backfitting

​

​

​

  • Update each component in turn given partial score .

20 of 39

GAME as a Library

  • Basic models implemented as building block .
    • Matrix factorization
    • Generalized Linear Model
    • …
  • New models can be directly composited by mixing existing building blocks.

21 of 39

Ex: GLM + MF

  • Predicting the response of user i on item j:

​

​

​

  • : GLM
  • : MF
  • : Expectation of response
  • : Link function

22 of 39

Future Direction: Wide & Deep Learning¹

1: Wide & Deep Learning for Recommender Systems

H.-T Cheng et al.

TensorFlow

23 of 39

GLMix¹: Fine-Grained GAME with Linear Components

1: GLMix: Generalized Linear Mixed Models For Large-Scale Response Prediction

X. Zhang et al., KDD2016

​

24 of 39

  • Jobs homepage
    • Ramped to serve 100% traffic (400 million LinkedIn members)
    • +20-30% lift in job application rate

​

24

GLMix @

25 of 39

  • Jobs homepage
    • Ramped to serve 100% traffic (400 million LinkedIn members)
    • +20-30% lift in job application rate
  • Article recommendation
    • +10-20% lift in CTR
  • Sponsored update (Ads)
    • Reduces model training time by 10x
  • People viewed this also viewed
    • +10-20% lift in offline experiments

​

25

GLMix @

26 of 39

Generalized Linear Model (GLM)

  • Predicting the response of user i on item j:

​

​

​

  • : Feature vector
  • : Coefficient vector
  • : Expectation of response
  • : Link function

27 of 39

GLM for Job Recommendation

27

​

  • Alice and Annie are about the same age, similar majors in college… (similar member features )
  • Alice likes to take more risks with start-ups
  • Annie likes more stable career just like her parents
  • GLM may return similar set of jobs to both

28 of 39

GLM for Job Recommendation

28

​

  • Alice and Annie are about the same age, similar majors in college… (similar member features )
  • Alice likes to take more risks with start-ups
  • Annie likes more stable career just like her parents
  • GLM may return similar set of jobs to both
  • Need more fine-grained modeling at different granularity to better personalize the model!

29 of 39

GLMix: Generalized Linear Mixed Model

  • Predicting the response of user i on item j:

​

​

​

  • Model coefficients with different granularities:
    • Per-user random effect coefficients
    • Per-item random effect coefficients

​

  • GLMix = GLM + per-user model + per-item model

30 of 39

GLMix for Job Recommendation

  • Global fixed effect model
    • Similarity between member profile and jobs profile, e.g. do the member skills and job skills look similar?

31 of 39

GLMix for Job Recommendation

  • Global fixed effect model
    • Similarity between member profile and jobs profile, e.g. do the member skills and job skills look similar?
  • Per-member random effect model
    • E.g. If a member has applied to a job with title = “software engineer”, we will boost “software engineer” jobs more in her results.

32 of 39

GLMix for Job Recommendation

  • Global fixed effect model
    • Similarity between member profile and jobs profile, e.g. do the member skills and job skills look similar?
  • Per-member random effect model
    • E.g. If a member has applied to a job with title = “software engineer”, we will boost “software engineer” jobs more in her results.
  • Per-job random effect model
    • E.g. If a job gets an apply with a member titled “software engineer”, we will boost this job more for members with this title.

33 of 39

Alice and Annie’s problem revisited

  • Per-user random effect coefficients for Alice:
  • Per-user random effect coefficients for Annie:
  • Alice and Annie now may have different job recommendations given their per-user coefficients.

34 of 39

Fine-Grained Modeling with GAME

35 of 39

Takeaways

  • GAME unifies and mixes different models into a principled additive model.
    • MF + GLM = RLFM
    • GLM + DNN = Wide & Deep Learning
    • ...

35

36 of 39

Takeaways

  • GAME unifies and mixes different models into a principled additive model.
    • MF + GLM = RLFM
    • GLM + DNN = Wide & Deep Learning
    • ...
  • GLMix is GAME with linear component that captures signal from different granularity
    • GLMix = GLM + Per-member model + per-item model + …

​

36

37 of 39

Takeaways

  • GAME unifies and mixes different models into a principled additive model.
    • MF + GLM = RLFM
    • GLM + DNN = Wide & Deep Learning
    • ...
  • GLMix is GAME with linear component that captures signal from different granularity
    • GLMix = GLM + Per-member model + per-item model + …
  • GAME is part of an open-source library

​

37

38 of 39

Hands-On Demo

39 of 39

©2014 LinkedIn Corporation. All Rights Reserved.

©2014 LinkedIn Corporation. All Rights Reserved.