1 of 24

18-663: Advanced Analytics and Machine Learning for Semiconductor Industry

�Spring 2025

​

Project 4

Eric, Hayden, Roman, Sid

ehu2, hsimon, rdkapur, ssingh4

​

2 of 24

Data Preprocessing

  • Data Merging and Cleaning
    • Merged wafer sort data (Die1: ~510k rows × 1,081 features; Die2: ~508k rows × 264 features) with final test data (805k rows) on "lot_key", "wafer_key", "die_x", "die_y"

​

​

​

    • Dropped unnecessary columns like "ECID", "wafer_key", "lot_key", "wf_key"
    • Standardized column names to remove special characters in feature names unsupported by model training
      • $, %, etc. were all replaced with underscores as a standard delimiter

​

  • Feature Engineering
    • Sorted data by categorical vs continuous by checking if > 10 unique values and if values were ints.
    • Imputed missing continuous values by replacing NaN with mean, imputed missing categorical values by replacing NaN with the most common category.

​

  • Imbalanced Train data
    • Addressed via scale_pos_weight to penalize misclassification of minority class.

3 of 24

Dimensionality Reduction

Attempted Techniques-

  • Univariate Screening: Initial feature reduction via Exensio ML-Ops Predictive Burn-in pipeline
  • PCA: Attempted to apply PCA to identified continuous features
    • Transformed features performed sub-par post model training (Low ROC-AUC values)
    • PCA process was also time consuming, hence due to time loss and low performance technique was abandoned
  • Model Feature Importance: Trained a number of models and extracted the top identified features for each to use for a reduced set
    • Die1: Top 100 features selected
      • Massive disparity in importance factor (top feature dominated performance)
    • Die2: Top 30 features selected
      • Importance more evenly distributed amongst all features

4 of 24

Significant parameters

Die 1:

  • Reduced to the 100 most significant parameters (full list in files)
  • Top 10 [DS1_X22813_17506_____1_____7, DS1_X17502_17504_____1_____7, DS1_X17471_17473_____1_____7, DS1_X17470_17472_____1_____7, DS1_X17485_17487_____1_____7, DS1_X17486_17488_____1_____7, die_y, DS1_X00006_8_____1_____7, DS1_X17479_17481_____1_____7, DS1_X15801_15803_____1_____7]

Die 2:

  • Reduced to the 30 most significant parameters (full list in files)
  • Top 10 [DS2_X22793_5294_____1_____3, DS2_X20134_2635_____1_____3, die_x, DS2_X21236_3737_____1_____3, DS2_X21981_4482_____1_____3, die_y, DS2_X22128_4629_____1_____3, DS2_X19459_1960_____1_____3, DS2_X22801_5302_____1_____3, DS2_X22802_5303_____1_____3]

​

Found by sweeping through varying thresholds of importance and finding the optimal and then accounting for generalization vs overfitting.

​

Full lists in Appendix.

5 of 24

Modeling Techniques

Data Splitting and Cross-Validation:

  • Stratified 80/20 Split:
    • Preserved the 1% minority class (failures) in both training and validation sets to avoid bias
    • Ensured proportional representation of "BIN_0" (defective) and "Normal" dies
  • 5-Fold Stratified Cross-Validation:
    • Evaluated model stability across diverse data subsets
    • Mitigated overfitting by averaging performance metrics (AUC, cost) across folds

​

​

Algorithm Selection: CatBoost:

  • Why CatBoost?
    • Native handling of categorical features (e.g. encoded integer parameters, positional data)
    • Robust to missing values (no explicit imputation required, critical for wafer test data)
    • Optimized for GPU training, enabling faster iterations on large datasets (~500k rows)
  • Advantages Over Alternatives (XGBoost/LightGBM):
    • Automatic handling of mixed data types (continuous + categorical)
    • Built-in support for class imbalance via scale_pos_weight

6 of 24

Modeling Techniques

Hyperparameter Tuning:

  • Key Parameters for CatBoost:
    • iterations=500: Balanced model complexity and training time
    • learning_rate=0.1: Ensured steady convergence without overshooting
    • depth=5: Restricted tree depth to prevent overfitting
    • eval_metric='AUC': Directly optimized for discriminative power

​

​

Threshold Tuning for Cost Sensitivity

  • Problem: Default threshold (0.5) minimizes classification error but not cost
  • Solution:
    • Tested 200 thresholds (0–1) on validation data.
    • Selected threshold that minimized custom cost function:
      • Die1 Cost = 10×FP + 15×FN
      • Die2 Cost = 4×FP + 9×FN
  • Outcome:
    • Optimal Thresholds: 0.86 for Die 1, 0.88 for Die 2 (reduced FP/FN trade-off)
    • Chose standard threshold 0.8 for both dies to mitigate overfitting

7 of 24

Evaluation Metrics & Cost Optimization

Die 1 Results:

  • Confusion Matrix:
    • TN = 391,255 | FP = 28 | FN = 23,125 | TP = 62
  • Cost Calculation:
    • $10 × FP + $15 × FN = $10×28 + $15×23,125 = $347,155 ($208,405)
  • AUC-ROC: 0.7265

​

Key Observations:

  • Extremely Low Recall (0.00) for BIN_0: Model failed to detect 99.7% of defective dies
  • High FN Cost Dominates: Majority of cost ($346,875) from undetected defective dies
  • Trade-off: Conservative threshold minimized FP (scrapping good dies) but led to catastrophic FN penalties

8 of 24

Die 1: AUC-ROC, Precision Recall Curve, Confusion Matrix

9 of 24

Evaluation Metrics & Cost Optimization

Die 2 Results:

  • Confusion Matrix:
    • TN = 386,133 | FP = 5,090 | FN = 13,882 | TP = 9,233
  • Cost Calculation:
    • $4 × FP + $9 × FN = $4×5,090 + $9×13,882 = $145,298 ($228,590)
  • AUC-ROC: 0.8495

​

Key Observations:

  • Improved Recall (0.40) for BIN_0: Detected 40% of defective dies, reducing FN cost by 40% compared to Die1
  • Balanced Trade-off: Higher FP (scrapped good dies) offset by significant reduction in FN (costly assembly failures)
  • AUC-ROC Surge: Better discriminative power (0.85 vs. 0.73 for Die1) due to optimized feature selection

10 of 24

Die 2: AUC-ROC, Precision Recall Curve, Confusion Matrix

11 of 24

Final Results on Test Dataset Die 1

$10 × FP + $9 × FN = $10×153 + $9×5607 = $51,993

​

12 of 24

Final Results on Test Dataset Die 1

13 of 24

Final Results on Test Dataset Die 2

$4 × FP + $15 × FN = $4 * 2233 + $15 * 3136 = $55,972

​

14 of 24

Final Results on Test Dataset Die 2

15 of 24

Appendix

Top 100 features histogram obtained for Die 1

​

16 of 24

Appendix

​

DS1_X22813_17506_____1_____7

DS1_X17502_17504_____1_____7

DS1_X17471_17473_____1_____7

DS1_X17470_17472_____1_____7

DS1_X17485_17487_____1_____7

DS1_X17486_17488_____1_____7

die_y

DS1_X00006_8_____1_____7

DS1_X17479_17481_____1_____7

DS1_X15801_15803_____1_____7

die_x

DS1_X07554_7556_____1_____7

DS1_X17496_17498_____1_____7

DS1_X15849_15851_____1_____7

DS1_X07584_7586_____1_____7

DS1_X17492_17494_____1_____7

DS1_X05917_5919_____1_____7

DS1_X17449_17451_____1_____7

DS1_X07602_7604_____1_____7

DS1_X17497_17499_____1_____7

DS1_X05923_5925_____1_____7

DS1_X16401_16403_____1_____7

DS1_X17445_17447_____1_____7

DS1_X07604_7606_____1_____7

DS1_X15773_15775_____1_____7

DS1_X16884_16886_____1_____7

DS1_X00478_480_____1_____7

DS1_X02686_2688_____1_____7

DS1_X07520_7522_____1_____7

DS1_X15767_15769_____1_____7

DS1_X16956_16958_____1_____7

DS1_X15297_15299_____1_____7

DS1_X04450_4452_____1_____7

DS1_X08154_8156_____1_____7

DS1_X15770_15772_____1_____7

DS1_X05920_5922_____1_____7

DS1_X03026_3028_____1_____7

DS1_X15916_15918_____1_____7

DS1_X01237_1239_____1_____7

DS1_X07889_7891_____1_____7

DS1_X16702_16704_____1_____7

DS1_X07526_7528_____1_____7

DS1_X16817_16819_____1_____7

DS1_X16816_16818_____1_____7

DS1_X15189_15191_____1_____7

DS1_X09295_9297_____1_____7

DS1_X09877_9879_____1_____7

​

​

DS1_X10899_10901_____1_____7

DS1_X05918_5920_____1_____7

DS1_X11271_11273_____1_____7

DS1_X00216_218_____1_____7

DS1_X16708_16710_____1_____7

DS1_X13858_13860_____1_____7

DS1_X05680_5682_____1_____7

DS1_X09894_9896_____1_____7

DS1_X01372_1374_____1_____7

DS1_X02855_2857_____1_____7

DS1_X05928_5930_____1_____7

DS1_X17156_17158_____1_____7

DS1_X05995_5997_____1_____7

DS1_X06008_6010_____1_____7

DS1_X13846_13848_____1_____7

DS1_X14210_14212_____1_____7

DS1_X00575_577_____1_____7

DS1_X07928_7930_____1_____7

DS1_X16175_16177_____1_____7

DS1_X16711_16713_____1_____7

DS1_X05933_5935_____1_____7

DS1_X14985_14987_____1_____7

DS1_X13854_13856_____1_____7

​

Top 100 features histogram obtained for Die 1

DS1_X16212_16214_____1_____7

DS1_X10872_10874_____1_____7

DS1_X14801_14803_____1_____7

DS1_X10531_10533_____1_____7

DS1_X03839_3841_____1_____7

DS1_X13548_13550_____1_____7

DS1_X13236_13238_____1_____7

DS1_X10877_10879_____1_____7

DS1_X07532_7534_____1_____7

DS1_X05924_5926_____1_____7

DS1_X17241_17243_____1_____7

DS1_X11247_11249_____1_____7

DS1_X00028_30_____1_____7

DS1_X00267_269_____1_____7

DS1_X04951_4953_____1_____7

DS1_X05959_5961_____1_____7

DS1_X14255_14257_____1_____7

DS1_X12790_12792_____1_____7

DS1_X07901_7903_____1_____7

DS1_X13167_13169_____1_____7

DS1_X16883_16885_____1_____7

DS1_X05932_5934_____1_____7

DS1_X13961_13963_____1_____7

DS1_X15964_15966_____1_____7

DS1_X13509_13511_____1_____7

DS1_X16400_16402_____1_____7

DS1_X07156_7158_____1_____7

DS1_X16709_16711_____1_____7

DS1_X07913_7915_____1_____7

DS1_X15779_15781_____1_____7

​

​

17 of 24

Appendix

Top 30 features histogram obtained for Die 2

​

18 of 24

Appendix

DS2_X22793_5294_____1_____3

DS2_X20134_2635_____1_____3

die_x

DS2_X21236_3737_____1_____3

DS2_X21981_4482_____1_____3

die_y

DS2_X22128_4629_____1_____3

DS2_X19459_1960_____1_____3

DS2_X22801_5302_____1_____3

DS2_X22802_5303_____1_____3

DS2_X22788_5289_____1_____3

DS2_X22789_5290_____1_____3

DS2_X20897_3398_____1_____3

DS2_X21282_3783_____1_____3

DS2_X19606_2107_____1_____3

DS2_X18741_1242_____1_____3

DS2_X22795_5296_____1_____3

DS2_X20113_2614_____1_____3

DS2_X22271_4772_____1_____3

DS2_X21238_3739_____1_____3

DS2_X22776_5277_____1_____3

DS2_X21242_3743_____1_____3

DS2_X21239_3740_____1_____3

DS2_X20119_2620_____1_____3

DS2_X20982_3483_____1_____3

DS2_X17608_109_____1_____3

DS2_X22120_4621_____1_____3

DS2_X20412_2913_____1_____3

DS2_X20426_2927_____1_____3

DS2_X21262_3763_____1_____3

​

Top 30 features histogram obtained for Die 2

19 of 24

Appendix

ROC and Precision Recall Curve for 5-fold LightGBM Die 1

​

20 of 24

Appendix

ROC and Precision Recall Curve for 5-fold LightGBM Die 2

​

21 of 24

Appendix

ROC and Precision Recall Curve for 5-fold XGBoost Die 1

​

22 of 24

Appendix

ROC and Precision Recall Curve for 5-fold XGBoost Die 2

​

23 of 24

Appendix

Confusion Matrices for Die 1

​

24 of 24

Appendix

Confusion Matrices for Die 2

​