Mohamed Ansar,�Applied Scientist @ Glance AI

E-COMMERCE SEARCH QUERY CLASSIFICATION: A TWO-LEVEL HIERARCHICAL APPROACH USING FINE-TUNED TRANSFORMERS

THE CHALLENGE OF E-COMMERCE SEARCH QUERY CLASSIFICATION

Why understanding user intent at scale is harder than it looks

The Classification Problem Landscape

JOURNEY AND EXPERIMENTS

Baseline Model → Hierarchical Classification Methods

LLM-Assisted Dataset Labeling for Quality & Coverage

Synthetic Data Generation for Rare Categories

MODEL ARCHITECTURES EXPLORED

End-to-End Fine-Tuning:

• Full transformer fine-tuned directly on all categories.
• Simple pipeline but sensitive to noisy labels and taxonomy complexity.

Fine-Tuned Encoder + Heads:

• Separate representation learning from classification.
• Encoder captures semantic features; dedicated heads handle category groups.

Independent Classifiers: Separate classifier per category group. Reduces inter-class interference and consistently outperforms a single global classifier.

Hierarchical Models: Preserve taxonomy structure across levels. Enable multi-label outputs and progressive refinement aligned with category hierarchy.

End-to-End & Encoder Approaches

Independent & Hierarchical Models

KEY FINDING: FINE-TUNED ENCODER + CLASSIFICATION HEADS OUTPERFORMS END-TO-END

❌ Underperformed due to noisy labels and high complexity. Single monolithic model struggled to generalize across 300+ subcategories. Taxonomy structure not preserved, leading to inconsistent hierarchy mapping.

✅ Separating representation learning from classification improved accuracy. Independent classifiers per category group outperformed a global classifier. Hierarchical structure preserved, enabling robust multi-label outputs.

End-to-End Fine-Tuning

Fine-Tuned Encoder + Classification Heads

MULTI-LABEL CLASSIFICATION IN A HIERARCHICAL SETTING

Handling Simultaneous Category Assignments

User queries can belong to multiple categories at once — e.g., "floral summer dress" spans Women's Clothing and Seasonal Collections. Our hierarchical structure naturally supports multi-label outputs, with dedicated classification heads per category group enabling flexible, independent assignments. This design prevents label interference across groups while preserving taxonomy constraints, resulting in more accurate and interpretable multi-label classification outcomes.

SOURCE-BASED CONFIDENCE THRESHOLDING

Real User Queries vs. Synthetic & LLM-Generated Data

Real user queries are often short, vague, and exploratory — making them inherently harder to classify with high confidence. Synthetic and LLM-generated data, while broader in coverage, differ in linguistic patterns and confidence levels. The system applies distinct confidence thresholds per data source to account for these differences, ensuring robust classification across all input types.

Balancing Precision and Recall Across Data Sources

By tuning thresholds independently for each source type, the model avoids over-committing to low-confidence predictions from noisy real queries while fully leveraging high-quality synthetic examples. This source-aware strategy improves both precision and recall, enabling the classifier to remain reliable even when input distributions shift significantly across user segments and query origins.

LEVERAGING LLMS FOR LABELING & SYNTHETIC DATA GENERATION

LLM-Assisted Data Enhancement

LLMs labeled new search logs with taxonomy validation, ensuring consistent category alignment. Synthetic examples were generated for rare subcategories with limited real-world data, improving coverage. Combined with symbolic data generation, this approach enhanced model generalization across low-traffic categories and bridged critical data gaps.

SYSTEM ARCHITECTURE OVERVIEW

Hierarchical Classification Pipeline — End-to-End Data Flow

From query input through transformer encoder and classification heads, to LLM-assisted labeling and source-based confidence thresholding, producing final hierarchical category outputs.

PRESERVING HIERARCHY IN CLASSIFICATION

Why Taxonomy Hierarchy Is Critical

Reflecting the label taxonomy in the model architecture ensures accurate and consistent category assignment across all levels. Hierarchy improves model interpretability by containing errors within logical category boundaries, preventing misclassifications from cascading across unrelated subcategories.

Multi-Level Classification & Taxonomy Constraints

A hierarchical structure enables progressive refinement — classifying at broad category level first, then drilling into subcategories. Taxonomy constraints are enforced during both training and inference, ensuring predictions remain coherent and aligned with the defined category structure across all ~300 subcategories.

Data quality and realistic labeling have a greater impact on performance than model complexity. Training data that closely resembles real user queries improves generalization significantly. Synthetic and LLM-generated data effectively bridge gaps in rare or low-traffic categories. Investment in data preparation and augmentation yields more reliable results than architectural complexity alone.

IMPORTANCE OF DATA QUALITY OVER MODEL COMPLEXITY

Lessons Learned: Data Beats Architecture

KEY TAKEAWAYS

Hierarchy Matters: Model architecture should reflect label taxonomy for accurate, consistent classification

Separate Tasks: Decoupling representation learning from classification heads improves overall performance

Data & Thresholding: Source-based confidence thresholding and high-quality data are critical to success

THANK YOU

FOR YOUR ATTENTION

Read the Full Article

mohamed.ansar@glance.com

Questions & Feedback Welcome