Content
summary Summary

OpenAI's GPT-4 now accepts visual input, and this capability will help it serve as a Virtual Volunteer for Be My Eyes to help the blind see.

Ad

Generative AI models like ChatGPT and DALL-E showed many practical use cases for artificial intelligence. While OpenAI's new GPT-4 can't generate images, it combines visual and language skills by adding the ability to accept images as input, boosting computer vision with the power of the large language model.

Be My Eyes is an app and service for people who are blind or have low vision that launched in 2015. Sighted volunteers guide and assist by watching through the user's phone, verbally describing what they can see, and answering questions.

With 6.3 million volunteers, Be My Eyes minimizes wait times and provide assistance as quickly as possible. The goal is to answer calls within seconds, but volunteers aren't always available, leading to delays of several minutes.

Ad
Ad

GPT-4 becomes a Virtual Volunteer

Since GPT-4 can understand the world through both text and images, it can act as a Virtual Volunteer for Be My Eyes and provide specialized help in ways that could be challenging for a human volunteer.

For example, with GPT-4 integration, translation is easy and doesn't require waiting for a person with knowledge of two particular languages. It could be used to navigate a train system, while traveling through a foreign country, browse websites and social media platforms, or do online shopping - "the possibilities are limitless, and we’re just getting started," Be My Eyes writes.

The user sends an image to GPT-4 in the enhanced Be My Eyes app. The response is in text form and can be read aloud by the app at the user's preferred speed. This could make getting help faster and more versatile.

Be My Eyes beta tester Lucy Edwards shared examples of how the AI-enhanced app works on her YouTube channel.

Be My Eyes GPT-4 availability

The GPT-4 integration is in beta testing for the next few weeks. The Virtual Volunteer will be rolled out to more users in the coming months, depending on feedback from early testers.

Recommendation

To request early access, you can join the waitlist in the Be My Eyes app. GPT-4 is available to both iPhone and Android users, and the AI feature will be accessible and free for all users eventually.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Be My Eyes is an app and service for people who are blind or have low vision, where sighted volunteers help them through their mobile phones.
  • The app now supports OpenAI's GPT-4 as a virtual volunteer, using the computer vision capabilities of the large language model.
  • Be My Eyes is available for iPhone and Android users, the GPT-4 integration is in beta testing and will be rolled out to more users in the coming months.
Sources
Alan loves to explore new technology and believes AR and VR could soon replace both phones and laptops as the best solution for work and play.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.