Single-image generation using multi-scale vector quantization with autoregressive prior

Single-image generative models aim to learn internal statistics of a single training image and synthesize diverse, realistic samples without using large-scale datasets. However, current models rely either on multi-scale generative adversarial network (GAN) pyramids or on diffusion models, both of which require controlling the continuous latent space of the training image. In this paper, we introduce a new single-image generative model that utilizes image-specific discrete latent representation by integrating vector quantization with a coarse-to-fine multi-scale pyramid. Vector quantization serves as a dictionary of high-frequency textural features, ensuring preservation of local textures and fine details that are typically lost in continuous latent space. To generate new image samples, our model trains an autoregressive prior over the quantized token maps, allowing image synthesis to be performed in a quantized latent space rather than directly in pixel space. Extensive quantitative and qualitative experiments on Places50, SIGD, and COCO50 datasets demonstrate that our model consistently achieves competitive performance in image quality, diversity, and computational efficiency, as measured by SIFID, NIQE, NIMA, LPIPS, and MS-SSIM, compared with existing models. We further demonstrate the versatility of our model through various image manipulation tasks, including paint-to-image, harmonization, editing, and outpainting. In conclusion, we show that vector-quantized latent modeling is an effective alternative to GAN- and diffusion-based approaches for high-quality and diverse single-image generation and represents a notable improvement over existing models. © 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.