At the Google Live event in Paris, the search giant announced several new AI features that enhance and expand AR capabilities.
AI has generated a lot of buzz lately, including its ability to create images that look like high-quality photographs or drawings. Similar systems can generate pages of text in a variety of styles and lengths in seconds, or write computer code to assist programmers.
Those are impressive capabilities, and Google will launch its chat AI, Bard, in the near future. Google is also leveraging AI in subtle ways to make AR effects more convincing and useful.
Google Lens is a perfect example; rather than typing a search, you can now take a photo to begin a search, translate text, and scan printed words into text that you can copy and paste on your phone.
Google Lens: Your camera is your keyboard
Google Senior Vice President Prabhakar Raghavan pointed out that the use of Google Lens has exploded, with more than 10 billion visual searches performed each month. As such, Google has been working to improve this technology that combines AI and AR.
Translation, one of the most useful Google Lens features, has been enhanced to make AR overlays more appealing. Previously, the AR translation appeared over a blocked-out background, hiding the foreign language text.
In the new version, Google Lens recreates the missing pixels with AI-generated graphics that blend seamlessly with the background. The result is a photo that appears as if it had always contained writing in your native language.
Lens recently received an upgrade called multisearch that allows you to refine a Lens result by adding keywords. A "near me" search finds similar products close to your current location. The "near me" feature launched first in the United States but will roll out to more countries in the coming months.
Google Maps becomes more immersive
Google's Immersive View renders detailed 3D models of select landmarks and cities in Google Maps thanks to AI analysis of billions of aerial photographs and 360-degree Street View images. The resulting graphics are almost photorealistic and allow you to fly virtually around objects and through cities via these digital recreations of reality.
This Google Maps technology was first announced in 2022 and included 360-degree views of 250 landmarks in 2022. Immersive View will soon be expanded to cover eight cities, including Los Angeles, San Francisco, New York, London, Tokyo, Amsterdam, Dublin, Florence, and Venice.
A related Google Maps feature, Indoor Live View, uses NeRF technology to let you visit select restaurants and other businesses virtually with your phone and works just like Street View.
Google is significantly increasing the number of locations that feature indoor AR navigation and immersive view. One thousand new airports, train stations, and shopping centers have been added. The emphasis is still on major metropolitan areas. Locations include New York, London, Melbourne, Sydney, Berlin, Frankfurt, Prague, Paris, Tokyo, Taipei, Singapore, Madrid, Barcelona, and Sao Paolo.
Google Live View eases travel
Google's Live View is an existing Google Maps feature that uses your phone's camera to show the location of places, overlaying visual labels in a live AR view. This will be incredibly useful when traveling through unfamiliar areas. Live View is rolling out now for Los Angeles, San Francisco, New York, London, Paris, and Tokyo. It's coming soon to Barcelona, Dublin, and Madrid.
Where available, you can tap the camera icon in the Google Maps search bar to enable Live View. You can tap on any pins that appear in the AR view to learn more about that location.
While restaurant reviews are helpful, finding your way around an airport is a necessity. Live View can superimpose arrows in your phone's viewfinder, so you won't get lost when navigating a busy airport or train station.
Google Art and Culture
The Google Arts and Culture app is packed full of information. It includes several AR features that make what might seem like dry topics more engaging.
Art selfie was Google's first on-device AI app from Google. It lets you snap a selfie or take a photo of a pet, then find similar people and animals in classic works of art.
A recent addition is Google's Woolaroo which supports 17 languages and can identify objects using your phone's camera. The Woolaroo web app overlays the words of recognized objects in the selected language. It's an immersive way to build up your vocabulary of foreign languages.
Google's Arts and Culture app can bring famous artwork, as well as archeological and scientific objects, into your home with its AR feature. The periodic table feature lets you view atoms at a room-filling scale via your phone. Google is making this learning tool available in many more languages soon, including German, French, Spanish, and more.
Overall, interest in AR models is increasing rapidly. Google reported that there has recently been an eight-times increase in the use of the AR features of Google Arts and Culture. The Google Arts and Culture app can be found in the iOS App Store and Google Play.
Google wrapped up the event by emphasizing that we no longer have to type text searches manually as more visual search options become available and expand to cover more locations in the future. This clearly sets the stage for the future launch of Google's AR headset or AR glasses. An XR headset, developed together with Samsung and Qualcomm, could integrate this visual information and search into everyday life even better than the smartphone screen.
