1 of 21

Kaggle Winner Presentation

Nguyen Pham Oanh Oanh

Dang Cong Khai

Le To Uyen

Le Dinh Chinh

2 of 21

2

Agenda

Kaggle Winner Presentation Template

3 of 21

3

  1. Background
  2. Summary
  3. Feature selection & engineering
  4. Training methods
  5. Important findings

Agenda

​

Kaggle Winner Presentation Template

4 of 21

4

Background

Kaggle Winner Presentation Template

5 of 21

5

Background

​

  • Le To Uyen
  • Major in Digital Business Management at VNUK, Institute for Research and Executive Education
  • Business Analyst intern at AIAIVN
  • Nguyen Pham Oanh Oanh
  • Major in Digital Business Management at VNUK, Institute for Research and Executive Education
  • Customer Service intern at Mstar Corp

Kaggle Winner Presentation Template

6 of 21

6

  • Dang Cong Khai
  • Major in Data Science at VNUK, Institute for Research and Executive Education
  • AI intern at Consen.AI, passionate about AI and Data

Background

​

  • Le Dinh Chinh
  • Major in Computer Science & Engineering at VNUK, Institute for Research and Executive Education
  • DevOps Engineer at AvePoint, jack of nothing, master of none <3.

Kaggle Winner Presentation Template

7 of 21

7

Summary

Kaggle Winner Presentation Template

8 of 21

8

  • Training method: Model stacking
  • Most important features:
    • BMI
    • FC data
  • Tools used: pandas, scikit-learn, xgboost, pytorch

Summary

​

Kaggle Winner Presentation Template

9 of 21

9

Features Selection/

Engineering

Kaggle Winner Presentation Template

10 of 21

10

Features Selection

​

Feature selection

  • Drop columns: study_site, handedness, parent_1_education, parent_2_education�

​

  • Keep the remaining columns: BMI, FC data, sex, ethnicity, race

Kaggle Winner Presentation Template

11 of 21

11

Features Selection

​

Feature engineering

  • Categorical data: ‘unknown’ value
  • BMI: median value as BMI is skewed
  • FC data: KNN imputer

Preprocessing

  • Standardize BMI column
  • One-hot encode: sex, ethnicity, race

Missing value

Kaggle Winner Presentation Template

12 of 21

12

Solution Overview

Kaggle Winner Presentation Template

13 of 21

13

Solution Overview

​

Ridge

BMI contributes significantly in predicting age

Kaggle Winner Presentation Template

14 of 21

14

Solution Overview

​

​

Gradient�Boosting

BMI and FC data contributes to age prediction for GB

Kaggle Winner Presentation Template

15 of 21

15

Solution Overview

​

Model stacking

12 base models: 'xgboost', 'linear regression', 'lasso', 'elastic net', 'ridge', 'kernel ridge', 'GB', 'LightGBM', 'SVR', 'LinearSVR', 'Random Forest', 'BayesianRidge'

​

Kaggle Winner Presentation Template

16 of 21

16

Solution Overview

​

Result

Submission

# of folds

# of models

Public score

Private score

Submission 1

5

12

1.60359

1.90900

Submission 2

10

12

1.61707

1.92115

Same approach as submission 2

5

12

1.61724

1.90845

Kaggle Winner Presentation Template

17 of 21

17

Important and Interesting Findings

Kaggle Winner Presentation Template

18 of 21

18

Important and interesting findings

What didn’t work for us

  • Using PCA to reduce FC dimensions, which disrupted �the fine-grained signal from FC data to predict age.

​

  • Using AutoEncoder to get the latent space of FC data

as input to predict age

Kaggle Winner Presentation Template

19 of 21

19

Important and interesting findings

​

What we learned

  • Remove explicitly irrelevant features (study_site, parent_education, etc) to avoid capturing noise as patterns
  • Stacking too many models might do more harm than good
  • Start with simple approaches, more complex ones later

Kaggle Winner Presentation Template

20 of 21

20

Question and Answer

Kaggle Winner Presentation Template

21 of 21

21

Kaggle Winner Presentation Template