Generative AI startup Midjourney may be entering the hardware space, having hired a head of hardware last December. What might he be working on?
In December, Ahmad Abbas joined Midjourney as head of hardware. Before that, he was a driving force behind Apple's Vision Pro: for more than five years, he helped develop Apple's mixed reality headset, most recently as hardware engineering manager. Abbas also spent about nine months at Elon Musk's brain chip startup Neuralink.
Abbas and Midjourney founder David Holz go way back: Abbas spent nearly four years as a hardware engineer at Leap Motion. The finger-tracking camera startup was founded by Holz in 2008.
Holz spent about 12 years as CTO of the company, which was acquired by Ultrahaptics in 2019 and now operates as Ultraleap. So both have a lot of hardware experience.
What is the Midjourney Orb?
Short answer: We don't know - and Holz and Abbas probably don't know yet either.
It's most likely a loose project name for the hardware strategy they're developing. For now, the Midjourney hardware team is focused on 3D data capture.
Midjourney is working on models for video and 3D generation and needs training data. Holz has said in the past that he is very optimistic that Midjourney will soon be able to generate good 3D assets.
Although nothing is known about the hardware itself, we can imagine what it will be used for: AI-generated 3D worlds and probably real-time generated video games.
Holz mentioned the project in early January 2024 in Discord conversations with Midjourney users, describing it as a device that could be used to generate and manage thousands of 3D rooms. He later confirmed on X that Midjourney "will make the Orb."
Holz also described his Midjourney image model as a "very slow game engine" and promised volumetric 3D worlds generated at 60 frames per second at some point in the future.
In a prediction from the summer of 2022, Holz said that in ten years, there will be a game console with an AI processor that generates games in real-time. Maybe it'll happen sooner than that.