Sony camera photographs "parallel universes" via DALL-E 2 app
A camera app connected to DALL-E 2 generates additional AI images of a real object in near real-time.
DALL-E 2 is best known for its ability to generate images that correspond to text input. However, the system can also generate variants of real-world subjects by, for example, showing an object from different perspectives or changing the content of an image. It is also capable of generating photorealistic motifs.
Photographing parallel universes
Photographer Nicholas Sherlock has taken advantage of DALL-E's ability and developed an unusual camera app: it sends a picture taken with the camera directly to DALL-E 2 via mobile Internet. The AI system then generates different versions of this picture and plays it back to the camera.
The photographer describes this new photographic process as a way of "peering into parallel universes". DALL-E 2 "dreams up new versions of them, revealing a world-that-might-have-been."
For simple subjects, the AI-based image variation works surprisingly well. Stones, trees, or plants, for example, are rendered realistically from different perspectives and in different lighting situations. It is sometimes difficult to distinguish between the original and the AI image.
However, in complex scenes like landscape shots, the AI images become more inaccurate and exhibit geometric errors and inappropriate blurring in the image composition. The transitions between image elements become messier.
Since DALL-E 2 is known to have problems with text, it replaces captions with fantasy language. The AI also replaces company logos or mascots with alternative versions. The result is quite entertaining.
Anyone looking for logos in the future could simply photograph suitable motifs on the street and have an image AI make similar suggestions.
Quantum Mirror App violates OpenAI guidelines
The DALL-E-2 app is available for Android-based Sony cameras (e.g. A7, A7R, A7 II, A7R II, but not the A7 III series or newer - list of possible compatible cameras) for free on Github. But beware: according to Sherlock, his app violates OpenAI's guidelines by being rated as web-scraping software. Violating the guidelines will result in an account suspension by OpenAI.
Sherlock's Quantum Mirror app is available on Github. On his project page, the photographer shows numerous other sample images. Sherlock plans to update his app as soon as OpenAI has released an API that complies with the new terms of use.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe nowRead on for the full picture.
Subscribe for hype-free coverage.
- Access to all THE DECODER articles.
- Read without distractions – no Google ads.
- Access to comments and community discussions.
- Weekly AI newsletter.
- 6 times a year: “AI Radar” – deep dives on key AI topics.
- Up to 25 % off on KI Pro online events.
- Access to our full ten-year archive.
- Get the latest AI news from The Decoder.