Feature Highlights

Where Segmentation-Free Mode Excels

This new feature works well in situations where clothing is more difficult to swap due to its shape, layers, or things like hair and accessories getting in the way. It makes virtual try-on more reliable by improving how different types of clothing and styles are handled. Some key improvements include:

• Better fit for long and loose clothing (e.g., tunics, layered tops)

• Improved support for bulky outerwear (e.g., coats, jackets)

• Easier handling of garments that cover the feet or have a wide bottom (e.g., mermaid dresses, long skirts)

• More accurate results when hair or accessories partially cover clothing

• Preserving head coverings and other cultural garments (e.g., hijabs) during swaps

• Better support for layered outfits where only some pieces are being changed

Limitations

While segmentation-free mode improves garment fit, it introduces a new limitation: a reduced ability to remove original garments entirely. Users may find that prior clothing elements remain visible in some transformations, which was not an issue in previous versions.

What Does “Segmentation-Free” Mean?

In virtual clothing try-on, image segmentation is the process of identifying and labeling each pixel in an image based on different categories such as clothing and body parts (e.g., shirt, pants, hands, face). This step is essential for separating garments from the wearer and ensuring complete clothing swaps. You can learn more about clothing and human segmentation in this blog post.

However, segmentation tools are not always perfect. Errors in the process can lead to noticeable issues, such as traces of the original garment appearing between strands of hair, unnatural transitions between tops and bottoms, and swapped garments unintentionally retaining the shape of the original clothing.

Segmentation-Free Mode eliminates the need for a separate segmentation step, allowing the try-on AI to directly process both the original outfit and the swapped garment in a single step. This results in smoother blending and a more natural fit. While we see this as a potential default mode in the future, current testing shows that although it enhances garment fitting, it does not yet fully remove the original swapped garment in all cases, leaving room for further refinement.

Examples, Best Practices and Trade-Offs

In this blog post, we will discuss the best practices and trade-offs when using this new feature. Understanding these aspects will help users make the most of segmentation-free mode while being aware of its strengths and limitations.

Better at Adding, Worse at Subtracting

Segmentation-free mode enhances the ability to add new garments while making it harder to completely remove existing ones. As a result, we recommend starting with models wearing outfits that are equal to or less in volume compared to the intended swap for the best results.

Strengths

In this section, we highlight scenarios where Segmentation-Free excels, delivering superior results.

Long Tops

The "Segmentation-Free" mode automatically adjusts to any top length, preserving the appearance of the bottom part. This resolves the challenge of fitting tops that extend to the knee or below, which was previously difficult even with the "Long Top" option turned on.

Fitting a longline t-shirt comparison

Fitting a kurta top, which extends below the knee

Bulky Outerwear

Fitting bulky clothing items like puffer jackets onto models wearing tight clothes can be challenging. The generated output is often restricted and doesn't expand to the necessary volume. Some users work around this by generating a model already wearing a similar item, but ideally, this shouldn't be necessary. With segmentation-free, even voluminous items can be fit onto models starting from a model in tight clothing.

Fitting a puffer jacket on a model wearing tight clothes comparison

Covering Feet & Wide Skirts

Segmentation-free mode makes it possible to properly fit long, flowing garments that cover the feet, such as mermaid dresses. Unlike the existing “Cover Feet” option, which may only cover the feet but struggles with large fabric volumes, segmentation-free mode effectively handles the fuller shapes found in prom dresses and wedding gowns.

Fitting an evening mermaid gown comparison

Hair Occlusions

One of the biggest advantages of allowing the try-on AI to handle segmentation is its ability to work directly with pixels that a segmentation model might miss, such as clothing pixels visible between strands of hair. While achieving perfect results for very fine strands remains challenging, this approach significantly improves hair preservation while seamlessly adapting the garment underneath.

Segmentation failure causing original garment color to leak into the try-on result—greatly improved with “Segmentation-Free” mode.

“Segmentation-Free” improves color handling between hair strands and enhances overall fit.

Preserving Head Coverings

Headscarves are an important part of many cultures and religious practices worldwide. “Segmentation-Free” improves the distinction between what should be swapped and what should remain untouched, ensuring a more accurate and natural fit in these scenarios.

Comparison of head covering distortion vs. “Segmentation-Free” preserving it properly.

Outfit Layering

While still a work in progress, Segmentation-Free shows promise in enabling more precise outfit layering. This includes swapping just the open jacket or only the shirt visible beneath it, rather than replacing the entire outfit.

Comparison of full jacket removal vs. “Segmentation-Free” replacing only the top.

Limitations

This section highlights scenarios where Segmentation-Free faces limitations, resulting in trade-offs that may lead some users to prefer the previous mode.

From Long to Short

In this example, both the standard mode and Segmentation-Free struggle, but in different ways. The original model wears a top that is longer than the try-on garment. The standard mode maintains the original length, resulting in an inaccurately long fit. Meanwhile, Segmentation-Free correctly adapts the fit but fails to remove the excess fabric. Since user preference in this case varies, we are still researching potential solutions and considering making this feature an optional setting rather than the default.

Both modes have drawbacks—standard mode keeps the garment too long, while “Segmentation-Free” fits it correctly but doesn’t remove the excess fabric.

From Bulky to Slim

As with the previous example, both modes have trade-offs. The standard mode completely removes the voluminous dress but distorts details like the shoes and blazer buttons. In contrast, Segmentation-Free achieves a perfect fit for the blazer and preserves the shoes but fails to fully remove the original dress.

A small amount of manual post-processing, such as erasing excess fabric in a photo editing tool, may be required for a flawless result. However, while fitting the garment accurately is extremely difficult without this AI tool, manually cleaning up artifacts is relatively simple. For users willing to do a bit of extra refinement, Segmentation-Free still offers significant advantages in this scenario.

Standard mode distorts shoes and buttons, while “Segmentation-Free” fits the blazer correctly but leaves some excess fabric.

Closing Words

The introduction of segmentation-free mode marks a significant improvement in clothing virtual try-on. While it may require a slight adjustment in how users approach the tool, making small changes to take advantage of its strengths and work around its weaknesses can lead to much better results than before. However, users who prioritize complete garment removal may still prefer the previous approach.

Segmentation-free mode is almost ready, but we are still refining it to make it a clear, no-brainer choice for most use-cases. Through ongoing testing, we are making improvements to ensure it delivers the best possible experience before release.

— Dan Bochman, FASHN Co-Founder