AI in practice

DreamGenerator is an AI camera with integrated Stable Diffusion prompts

Matthias Bastian

Kyle Goodrich

AI image models can generate new images - or modify existing ones. DreamGenerator shows how this could work in a camera.

Unlike conventional cameras, DreamGenerator lets you choose the world in which the next photo will be taken: Heaven or hell, the Middle Ages, underwater or outer space, there are many variations. Thirty themes are pre-programmed.

Once a photo is captured, it is instantaneously transformed into a new image using the pre-selected theme. The fundamental characteristics of the photo are preserved, such as an individual's posture and facial features, or the perspective from which a car is photographed in a parking lot. The only alteration is that, following the capture of the photo, the car depicted becomes a new Ferrari, in place of an old Honda.

The DreamGenerator turns an old Honda into a new Ferarri. The name of the device seems aptly chosen. | Image: Kyle Goodrich

"The generated images reference the composition and pose of the original photo, ensuring that key elements are retained while also adding in new imaginative touches," writes developer Kyle Goodrich.

Of course, this has nothing to do with authentic photography. But Goodrich says he's primarily interested in simplifying the complex prompting process of systems like Stable Diffusion. You could do that with a smartphone app, obviously, but Goodrich says he prefers the simplicity of a point-and-shoot camera.

ControlNet allows Stable Diffusion fine-tuning with minimal data

For image generation, Goodrich uses a combination of the open-source Stable Diffusion image AI and ControlNet, a simple fine-tuning method that greatly enhances Stable Diffusion's image-to-image capabilities.

Here, fine-tuning for a given subject is done with tiny data sets, such as a single photograph. Like Stable Diffusion, ControlNet is available as free open-source software and even runs on smartphones.

In the video below, Goodrich, who is an AR product designer at Snapchat, shows off a prototype of the hardware and software. He doesn't mention a retail version, so those who want the AI camera will likely have to build it themselves (or build it as an app for your smartphone).

Sources:
Twitter