The problem
Product categorization is harder when the meaning of an item depends on its scene. Using Pinterest Shop the Look–derived fashion and home decor datasets, our team explored whether product geometry and scene-level context could improve automated category tagging.
The approach
Derived bounding-box area, diagonal, center position, relative size, scene density, diversity, and other context-aware features.
Compared simple classifiers, including logistic regression and Naive Bayes, before building tree ensembles.
Evaluated Random Forest and XGBoost with engineered and latent factor features. Applied stratified splitting and tuned parameters with GridSearchCV.
A result worth showing
In the fashion dataset, adding latent factor features to a Random Forest achieved the best reported accuracy and F1 score. Results also highlighted class imbalance and the difficulty of generalizing across diverse product categories.
Why it matters
Tagging is a foundational layer of search, discovery, and recommendations. Better contextual features can help systems organize visual products more meaningfully, while thoughtful evaluation prevents headline accuracy from masking weaker performance for minority classes.
Explore the work
Read the original project materials behind this case study.
Book Recommendation & Prediction