I’ve been wanting to turn my face into those city-hopping, suit-swapping images, so I messed around with DreamBooth training on SDXL. Used SDXL 1.0 Base as the base model—turns out this foundation is still the most solid for realistic stuff.
Biggest lesson was the dataset. My training images were only mid-tier quality—backgrounds kept repeating, clothes were repetitive, and I was missing full-body poses. Ended up with overfitting and poor generalization; every generation kept pulling from those few backgrounds in the training set. For regularization images, I picked high-quality real person photos—you can’t cut corners on this step.
Another trap was epoch checkpoints. I saved from epoch 150, and the outputs were clearly overtrained. If you want to save GPU, Kaggle’s free notebook can handle it—just calculate your steps and set the save frequency accordingly. Overall it’s not hard, just a lot of small details.