使用diffusion mode生成synthetic images,用于enhance traning data:
1. FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models (NurIPS 2024)
Challenges:
- Generating in-domain images
- Generating high-quality images
- Generating hard samples to improve model generalization ability
How to solve the challeges:
- FreestyleNet, which can generate high-fedility images conditioned on semantic masks
- Incorrectly synthesized regions or images will exhibit significant losses, if evaluated under a model pre-trained on real images ---- > if the loss of a synthetic pixel surpasses the average loss of its corresponding class by a certain margin, it will be marked as a noisy pixel and ignored during loss computation
- with recorded class-wise average losses, it can calculate the overall loss of a semantic mask, which can represent its global hardness. It will generate more samples for those hard semantic masks and otherwise the opposite
短语:“It is even on par with”