OpenAI has rolled out the new feature “Images in ChatGPT,” which brings direct image generation capabilities into the ChatGPT platform. The new GPT-4o model enables users to generate images during conversations which represents a major advancement in AI-generated content creation.

The latest feature is accessible through all ChatGPT subscription levels, including Plus, Pro, Team, and the free tier, to extend sophisticated image generation capabilities. Users of the free tier system face the same limitations as DALL-E 3 while generating about three images each day, but OpenAI spokesperson Taya Christianson stated that these restrictions could change in response to usage demand. DALL-E enthusiasts will maintain access through a specially created custom GPT interface.

The research lead at OpenAI, Gabriel Goh, described GPT-4o as a transformative “omnimodal” model that processes multiple data formats, such as text and images, together with audio and video. The model now exhibits improved “binding” capabilities, which resolve a longstanding issue in AI image generation. Where past models often mixed up object attributes, GPT-4o processes 15 to 20 objects accurately without color or shape confusion.

The system’s advanced text rendering stands out as a key advancement. AI-generated images have historically displayed distorted or meaningless text elements. Goh explained that developing this system required extended iteration, which took numerous months to perfect. Despite ongoing difficulties in perfecting text rendering for small text, the team succeeded in developing reliable text consistency throughout images.

This system departs from typical diffusion model designs in image generators to use an autoregressive method instead. The sequential generation of images from left to right and top to bottom based on text generation techniques helps improve text rendering and binding abilities.

In their presentation, OpenAI demonstrated the system’s capabilities across various domains by showing how it can produce scientific diagrams of Newton’s prism experiment with detailed labels as well as multi-panel comics with character consistency and dialogue, and informational posters with correct text content. Demonstrations included practical uses such as the creation of transparent background images for stickers and restaurant menus, and logos.

Jackie Shannon, who leads ChatGPT’s multimodal product team, highlighted the system’s capability to utilize global knowledge. When I create an image, my skills limit me, but I incorporate the world knowledge I have gained. The model incorporates world knowledge into its function, which allows users to request images of Newton’s prism experiment without needing to provide any background information.

OpenAI believes that the improvements in quality and capabilities make the extended time needed for image generation worthwhile. Although we can work on decreasing latency, the advanced quality of these images and their ability to use extensive world knowledge compensate for the extra waiting time.

Key Features and Safeguards Implemented by OpenAI:

Enhanced Binding: GPT-4o accurately tracks connections between 15 to 20 objects while minimizing color and shape recognition errors.

Improved Text Rendering: Through precise development, OpenAI has achieved more reliable text rendering in generated images, which solves a frequent challenge faced in AI development.

Autoregressive Approach: Through its sequential image generation system, the platform potentially improves how it manages text and objects.

Robust Safeguards: OpenAI established protocols to stop watermark deletion while blocking sexual deepfakes and declining requests for CSAM.

C2PA Metadata: The standard C2PA metadata accompanies all generated images to signify their origin from OpenAI.

User Ownership: Users have ownership rights to their generated images as long as they operate within established usage policies.

OpenAI emphasized its deployment of strong protective measures to tackle potential misuse concerns. No system achieves perfection for this type of task, but our safeguards are always improving, and this represents our initial approach according to Shannon. Users who generate images through ChatGPT own these images while having the freedom to use them according to OpenAI’s usage policies.

Through “Images in ChatGPT,” OpenAI improves its main product functionality and establishes a new benchmark for powerful AI image generation while actively managing potential risks.