Draw Things is an app that brings Stable Diffusion to the iPhone. The AI images are generated locally, so you don't need an Internet connection.
Thanks to Stable Diffusion's open-source code, AI images can also be generated on a powerful PC rather than in the cloud. Developer Liu Liu has now packaged Stable Diffusion's algorithm into an iPhone app.
The app does not require an Internet connection after downloading the roughly two-gigabyte resources. All images are generated directly on the iPhone.
According to the developer, the app uses only 50 percent of the available computing power. Implementing Stable Diffusion on the iPhone was "easier than I thought," Liu Liu writes.
Memory is the bottleneck
The bottleneck is memory. "The biggest challenge is running the app on iPhones with 6 GB of RAM," Liu Liu writes. 6 GB would sound like a lot, but iOS automatically kills apps if they use more than 2.8/6 GB or more than 2/4 GB of RAM, the developer said.
To solve this problem, Liu Liu broke down Stable Diffusion into its components. He examined exactly how much RAM the generator requires and how the demand can be reduced in each phase.
The model has 4 parts: a text encoder that generates text feature vectors to guide the image generation. An optional image encoder to encode image into latent space (for image-to-image generation). A denoiser model that slowly denoise out a latent representation of an image from noise. An image decoder to decode the image from that latent representation. The 1st, 2nd, and 4th models need to run once during inference. They are relatively cheap (around 1GiB max). The denoiser model’s weights occupy 3.2GiB (in full floating-point) of the original 4.2GiB model weights. It also needs to run multiple times per execution, so we want to keep it in RAM longer.
Liu Liu
The intermediate stages involved in generating an image would be particularly resource-hungry. Liu Liu predicts that with further optimization, runtime could be reduced by 30 percent and memory usage by about 15 percent.
Draw Things comes with different models
Liu Liu has managed to integrate a remarkable number of features into the app. Draw Things currently supports the following models:
- Stable Diffusion 1.4
- Inpainting (Stable Diffusion 1.5)
- Anime (Waifu Diffusion 1.3)
- Classic Animation
- Modern Disney
- Arcane
- Cyberpunk Anime
- 3D Model
Apart from the desired model, users can make further settings for resolution (up to 512 x 768 pixels), number of steps (10 to 130), guidance scale, strength, seed and sampler (DDIM, PLMS, Euler Ancestral, DPM++ 2M Karras). Especially handy is the version overview, where every change can be viewed in detail.
Draw Things runs on iPhone 11 and higher
Draw Things is now available for free via the App Store and comes without advertising. An iPhone 11 or higher and iOS 15.4 are required.
I tried Draw Things on an iPhone 14 Pro Max: A 512 x 512 pixel image with 30 steps takes about a minute, and with a maximum of 130 steps, the process lengthens to about five minutes. With these settings, you can already achieve decent results. The heat development is noticeable on the iPhone, but the device doesn't get hot.
