Models & resources
Imagen
Imagen is a text-to-image diffusion model developed by the Google Research Brain Team, designed to achieve an unprecedented degree of photorealism and a deep level of language understanding.
What is Imagen?
Imagen is a text-to-image diffusion model developed by the Google Research Brain Team, designed to achieve an unprecedented degree of photorealism and a deep level of language understanding.Its key discovery is that generic large language models (e.g., T5), pretrained on text-only corpora, are surprisingly effective at encoding text for image synthesis: scaling up the language model boosts both sample fidelity and image-text alignment more than increasing the size of the image diffusion model.
Technically, Imagen uses a large frozen T5-XXL encoder to map input text into embeddings.A conditional diffusion model then generates a 64x64 image, which is further upsampled to 1024x1024 resolution via text-conditional super-resolution diffusion models.This cascaded approach, combined with a new Efficient U-Net architecture, ensures high computational and memory efficiency.
In benchmarks, Imagen achieves a state-of-the-art FID score of 7.27 on the COCO dataset without ever training on it.Through DrawBench, a comprehensive new benchmark introduced by the team, human raters consistently prefer Imagen over recent methods like DALL-E 2 and Latent Diffusion Models in side-by-side comparisons for both image quality and text alignment.
Key features
- Deep Language UnLeverages large pretrained frozen text encoders (like T5-XXL) to accurately comprehend complex prompts, spatial relations, and rare words.
- Photorealistic QGenerates high-fidelity 1024x1024 images with exceptional detail using cascaded diffusion models and a new thresholding diffusion sampler.
- Superior Text-ImHuman raters in the DrawBench benchmark consistently prefer Imagen's output for its accurate adherence to textual descriptions compared to other models.
- Efficient ArchitUtilizes a new Efficient U-Net architecture that is more compute-efficient, memory-efficient, and converges faster than previous designs.
- State-of-the-ArtAchieves a record-breaking FID score of 7.27 on the COCO dataset, despite not being trained on COCO data.
How to use it
- 1.Visit the official Imagen research page to explore the model architecture and technical details.
- 2.Read the published research paper to understand the integration of T5 encoders with diffusion models.
- 3.Review the DrawBench benchmark results to compare Imagen's performance against other models on complex prompts.
- 4.Stay updated on Google Research announcements regarding responsible externalization frameworks and potential code releases.
- 5.Refer to related literature to understand the broader advancements of diffusion models in image generation.
Where it fits
- Academic ResearcUsed to explore the encoding capabilities of large language models in visual generation tasks and their impact on text-image alignment.
- Benchmark EvaluaServes as a reference standard in comprehensive benchmarks like DrawBench to evaluate the performance of next-generation text-to-image models.
- High-Fidelity AsGenerates photorealistic images from detailed text descriptions, suitable for scenarios requiring high-quality visual content.
Pricing and free quota
Imagen is currently presented as a research project by Google Research and does not offer public commercial APIs or subscriptions; availability is subject to the official site.
Who it is for
- AI Researchers
- Computer Vision Engineers
- Machine Learning Developers
- Digital Art Creators
FAQ
Is Imagen code or demo publicly available?
Due to concerns about misuse and societal impact, Google Research has decided not to release code or a public demo at this time, pending a framework for responsible externalization.
How does Imagen differ from models like DALL-E 2?
Imagen is simpler as it does not need to learn a latent prior; instead, it relies on large frozen language models for text encoding, achieving better results in COCO FID and human evaluations.
What is DrawBench?
DrawBench is a comprehensive and challenging benchmark introduced by Google Research to systematically test text-to-image models on compositionality, cardinality, spatial relations, and more.