Mohamed Ansar,�Applied Scientist @ Glance AI

E-COMMERCE SEARCH QUERY CLASSIFICATION: A TWO-LEVEL HIERARCHICAL APPROACH USING FINE-TUNED TRANSFORMERS

THE CHALLENGE OF E-COMMERCE SEARCH QUERY CLASSIFICATION

Why understanding user intent at scale is harder than it looks

The Classification Problem Landscape

JOURNEY AND EXPERIMENTS

Baseline Model → Hierarchical Classification Methods

LLM-Assisted Dataset Labeling for Quality & Coverage

Synthetic Data Generation for Rare Categories

MODEL ARCHITECTURES EXPLORED

End-to-End Fine-Tuning:

• Full transformer fine-tuned directly on all categories.
• Simple pipeline but sensitive to noisy labels and taxonomy complexity.

Fine-Tuned Encoder + Heads:

• Separate representation learning from classification.
• Encoder captures semantic features; dedicated heads handle groups.

Independent Classifiers:

• Separate classifier per category group.
• Reduces inter-class interference and outperforms a global classifier.

Hierarchical Models:

• Preserve taxonomy structure across levels.
• Multi-label outputs and progressive refinement aligned with hierarchy.

End-to-End & Encoder Approaches

Independent & Hierarchical Models

KEY FINDING: FINE-TUNED ENCODER + CLASSIFICATION HEADS OUTPERFORMS END-TO-END

❌ Underperformed due to noisy labels and high complexity. Single monolithic model struggled to generalize across 300+ subcategories.

❌ Taxonomy structure not preserved, leading to inconsistent hierarchy mapping.

✅ Separating representation learning from classification improved accuracy.Independent classifiers per category group outperformed a global classifier.

✅ Hierarchical structure preserved, enabling robust multi-label outputs.

End-to-End Fine-Tuning

Fine-Tuned Encoder + Classification Heads

Macro F1 Score Improvement

Same data. Structural changes only.

MULTI-LABEL CLASSIFICATION IN A HIERARCHICAL SETTING

Handling Simultaneous Category Assignments

• User queries can belong to multiple categories at once, e.g., "polo shirt" spans both shirt and t-shirt (1 query, 2 subcategory labels - sits between both by design). The hierarchical structure naturally supports multi-label outputs, with dedicated classification heads per category group enabling flexible, independent assignments.
• This design prevents label interference across groups while preserving taxonomy constraints, resulting in more accurate and interpretable multi-label classification outcomes.

SOURCE-BASED CONFIDENCE THRESHOLDING

Real User Queries vs. Synthetic & LLM-Generated Data

• Real user queries are often short, vague, and exploratory — making them inherently harder to classify with high confidence.
• Synthetic and LLM-generated data, while broader in coverage, differ in linguistic patterns and confidence levels.
• The system applies distinct confidence thresholds per data source to account for these differences, ensuring robust classification across all input types.

Balancing Precision and Recall Across Data Sources

• By tuning thresholds independently for each source type, the model avoids over-committing to low-confidence predictions from noisy real queries while fully leveraging high-quality synthetic examples.
• This source-aware strategy improves both precision and recall, enabling the classifier to remain reliable even when input distributions shift significantly across user segments and query origins.

LEVERAGING LLMS FOR LABELING & SYNTHETIC DATA GENERATION

LLM-Assisted Data Enhancement

• LLMs labeled new search logs with taxonomy validation, ensuring consistent category alignment.
• Synthetic queries generated for rare subcategories with limited real-world data, improving coverage.
• Combined with symbolic data generation, this approach enhanced model generalization across low-traffic categories and bridged critical data gaps.

SYSTEM ARCHITECTURE OVERVIEW

Flow from query input through transformer encoder and classification heads, to LLM-assisted labeling and source-based confidence thresholding, producing final hierarchical category outputs.

PRESERVING HIERARCHY IN CLASSIFICATION

Why Taxonomy Hierarchy Is Critical

• Reflecting the label taxonomy in the model architecture ensures accurate and consistent category assignment across all levels.
• Hierarchy improves model interpretability by containing errors within logical category boundaries, preventing misclassifications from cascading across unrelated subcategories.

Multi-Level Classification & Taxonomy Constraints

• A hierarchical structure enables progressive refinement, classifying at broad category level first, then drilling into subcategories.
• Taxonomy constraints are enforced during both training and inference, ensuring predictions remain coherent and aligned with the defined category structure across all ~300 subcategories.

• Data quality and realistic labeling have a greater impact on performance than model complexity.
• Training data that closely resembles real user queries improves generalization significantly.
• Synthetic and LLM-generated data effectively bridge gaps in rare or low-traffic categories. Investment in data preparation and augmentation yields more reliable results than architectural complexity alone.

IMPORTANCE OF DATA QUALITY OVER MODEL COMPLEXITY

Lessons Learned: Data Drives Performance

KEY TAKEAWAYS

Hierarchy Matters: Model architecture should reflect label taxonomy for accurate, consistent classification

Separate Tasks: Decoupling representation learning from classification heads improves overall performance

Data & Thresholding: Source-based confidence thresholding and high-quality data are critical to success

Extensibility and Future Scope

THANK YOU

FOR YOUR ATTENTION

Read the Full Article

mohamed.ansar@glance.com

Questions & Feedback Welcome