Fewer returns when shopping online would benefit both consumers and sellers. A new model from Amazon is designed to realistically depict how clothes might look on your body or furniture in your room.
Amazon researchers have unveiled "Diffuse to Choose" (DTC), an AI tool that allows customers to virtually place clothing or furniture from an online store on their bodies or in their rooms. The idea is to give them a better idea of the fit and spatial effect in real-time.
Amazon's DTC uses inpainting model
DTC is a diffusion-based image inpainting model for "virtual trial and error". The technology provides detailed and semantically coherent fusion with realistic lighting and shadows, according to the paper.
DTC uses two separate networks to process the product image. One network focuses on the overall image, while the other focuses on the product. This helps preserve the fine details of the product, such as the texture of the sofa's fabric. According to the researchers, DTC outperforms existing methods without the need for product-specific training.
The model was tested on both Amazon and public data sets.
However, there are still some limitations, mainly related to the Stable Diffusion base. For example, text can be rendered poorly or the pose of people in photos can be changed. Although the latter problem could possibly be solved by further training, the researchers deliberately chose to make the model more broadly applicable (e.g. for interior design) rather than training it in specific disciplines.
A new standard in e-commerce?
For consumers, DTC promises a more personalized shopping experience and could help them make more informed purchasing decisions. Retailers, on the other hand, will benefit from a new level of customer loyalty and potentially lower return rates if products can be better displayed before purchase.
While Google has also developed a virtual try-on tool for online retail that focuses on the realistic presentation of clothing, Amazon's DTC aims to work across product categories.
Amazon plans to release the code and a demo of DTC soon. If the research lives up to its promise, tools like DTC could herald the next stage in the evolution of e-commerce.