K-EmoPhone Dataset

“Emotion & Stress Tracking”

Minhajur Rahman Chowdhury Mahim

2025.06.13

Final Submission

Overview

• Objective: Predict user stress using smartphone and wearable data.
• Dataset: K-EmoPhone (2,619 samples)
• Approach: Feature engineering, selection, tuning, and modeling
• Challenge: High-dimensional tabular data (5,589 features)
• Keeping 5-fold StratifiedGroupKFold validation for consistency

​

​

{'feat_time': 16,

'feat_dsc': 55,

'feat_current_sensor': 84,

'feat_current_ESM': 1,

'feat_ImmediatePast_sensor': 416,

'feat_ImmediatePast_ESM': 0,

'feat_today_sensor': 2496,

'feat_today_ESM': 6,

'feat_yesterday_sensor': 2496,

'feat_yesterday_ESM': 6,

'feat_sleep': 2,

'feat_pif': 11}

Assignment 1 - Feature Combinations

• Current Baseline: Feat_baseline
• Time related features (feat_time), duration since last change for categorical data (feat_dsc), current sensor features (feat_current_sensor), and immediate past sensor features (feat_ImmediatePast_sensor)
• Goal: Test performance impact of adding/removing groups
• All other settings rather than feature combinations will remain unchanged

Feature Combinations

Assignment 2 - Feature Selection

• Methods Tried in the Baseline:
• - LASSO (threshold = 0.005, mean)
• - XGBoost embedded importances

​

• Goal: Retain predictive, non-redundant features
• Using the baseline feature groups and the best feature combinations from Assignment 1

Assignment 2 - Feature Selection

Feature Selection - LASSO (Mean AUC > 0.59)

Feature Selection - SHAP (Mean AUC > 0.60)

Put all features in shap:

• Captures contributions from all possible features.�
• Problem: Running SHAP on full data before split → test data influences selection.
• Solution:
• Used StratifiedGroupKFold (n=5, same seed) to match final CV splits.
• Collected only training indices across folds.

Merged training data → fit model → compute SHAP.

• Selected top-N features from training data only.

Feature Selection - Random Forest (Mean AUC > 0.60)

​

• Captures non-linear interactions between features and the target.
• Ranks features based on their contribution to reducing prediction error across many trees.�

Assignment 3 - Hyperparameter Tuning

• Using Hyperopt (Bayesian optimization)
• uses Bayesian optimization to explore areas of the parameter space
• Parameters Tuned: max_depth, learning_rate, subsample, colsample_bytree, reg_alpha, reg_lambda

​

​

• Goal: Optimize XGBoost for best performance (keeping the baseline model)
• Dataset: Using Top 60 features got using SHAP (best result so far)

Assignment 3 - Hyperparameter Tuning: Hyperopt (Same)

​

space = {

'max_depth': hp.choice('max_depth', list(range(3, 10))),

'min_child_weight': hp.quniform('min_child_weight', 1, 10, 1),

'subsample': hp.uniform('subsample', 0.6, 1.0),

'colsample_bytree': hp.uniform('colsample_bytree', 0.6, 1.0),

'gamma': hp.uniform('gamma', 0, 5),

'learning_rate': hp.loguniform('learning_rate', np.log(0.01), np.log(0.3)),

'n_estimators': hp.quniform('n_estimators', 50, 300, 10),

'reg_alpha': hp.uniform('reg_alpha', 0, 1),

'reg_lambda': hp.uniform('reg_lambda', 0, 1),

'random_state': 42

}

​

{'colsample_bytree': 0.8463935920953023, 'gamma': 0.2144097629496495, 'learning_rate': 0.0736917612203914, 'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 280, 'reg_alpha': 0.4775914535854724, 'reg_lambda': 0.6862888539325457, 'subsample': 0.7290628198195405, 'random_state': 42}

​

Mean AUC: 0.566

Assignment 4 - Deep Learning

• Model: TabNet (attention-based DL for tabular data)
• Compared to tuned XGBoost
• Goal: Use same feature set as top-performing XGBoost setup from previous assignments

​

Comparing with XGBoost:

• Macro AUC-ROC
• Feature combinations

TabNet Benchmark Results

Feature Selections

• SHAP Top 60: Selected based on SHAP (EvXGBoost), having best pre-tuning AUC.
• All Features: Deep learning handles feature selection implicitly via regularization (neuron dropout/pruning).

�

Hyperparameter Tuning

space = {

"n_d": hp.choice("n_d", [32, 64, 128]),

"n_a": hp.choice("n_a", [32, 64, 128]),

"n_steps": hp.choice("n_steps", [3, 5, 7]),

"gamma": hp.uniform("gamma", 1.0, 2.0),

"lambda_sparse": hp.loguniform("lambda_sparse", np.log(1e-6), np.log(1e-2)),

"lr": hp.loguniform("lr", np.log(1e-3), np.log(5e-2)),

}

Assignment 5 - Final Model

• Features: The shap top 60 features from assignment 2
• Soft voting ensemble: evXGBoost and TabNet
• Averages or weights the predicted probabilities from multiple models instead of using their final class predictions.

Features Chosen by SHAP

Discussion

• 26 out of 50 features involved #ImmediatePast features
• Recent historical context

​

• Context-rich features like #ImmediatePast15, #TodayAfternoon, and #YesterdayNight appear frequently
• Temporal dynamics does matter.

​

• ESM data rather act best as guiding factor (from assignment 1)

​

• A mix of objective signals (sensors) and contextual data (ESM, location labels) are important�

Code Review

• Deterministic Results� Random seeds fixed, model states saved to ensure reproducibility.�
• Clarity Through Comments� Well-placed inline comments and markdown cells improve readability.�
• Fair Model Comparison� All models and baselines evaluated using identical settings and metrics.�
• Clean Code Style� Clear variable names and consistent formatting enhance maintainability.

​

Snippets

Retraction From Final Presentation

• There was an error in doing SHAP based feature selection
• Eventually the SHAP was mistakenly done on the entire dataset
• Correct measures have been taken place to ensure no data leakage
• It has been mentioned in the SHAP slides

​

• Some performance metrics have been changed due to this change

Resources

1. Kang, S., Choi, W., Park, C.Y., Kim, J.H., Lee, J., & Lee, U. (2023). K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention Labels. Scientific Data, 10, 351.
2. Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288.
3. Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794).
4. Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017).
5. Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5–32.
6. Bergstra, J., Yamins, D., & Cox, D. D. (2013). Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), 115–123.
7. Arik, S. Ö., & Pfister, T. (2021). TabNet: Attentive Interpretable Tabular Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 6679–6687.

Thank you