Google is integrating AI into just about all of its products, including experimental applications like Gameface, a facial recognition mouse, and Tailwind, a private language model service that lets users chat with their notes.
In addition to its major AI efforts with chatbots, search, and the PaLM 2 language model, Google also unveiled a number of smaller AI applications and experiments. Two of these are Project Gameface, which uses a webcam and machine vision to track your face, and Project Tailwind, an AI notebook that learns from Google Drive documents.
Project Gameface is a face-tracking mouse
In Project Gameface, Google uses the webcam to track the face for mouse-like interaction with a computer. According to Lead AI Advocate Laurence Moroney, Google uses a number of different AI models that project and track a total of 468 tracking points on the face.
The movements of these tracking points are translated into mouse actions through a native Windows application. This allows the user to control the computer with facial movements.
For example, pulling the right corner of the mouth to the right will make the character in a game walk to the right. Exactly how the face controls the computer can be fine-tuned through an extensive options menu.
Currently, the software supports six facial movements, which is still quite limited and not suitable for complex games. Gameface is primarily designed for people with disabilities. The software is open source and available on Github with installation instructions.
Tailwind: AI notebook based on a private language model
Google calls Tailwind an "AI-first notebook." It uses a private language model trained on your documents and notes stored in Google Drive. You can then interact with your content through a chat interface, asking for specific information, inspiration, or summaries. Imagine ChatGPT, but it knows everything about our specific topics. Google is using its own PaLM 2 API for Tailwind.
According to Google, the prototype is still in its early stages and was initially designed and tested for students. However, the principle behind Tailwind is relevant to many audiences, from authors to financial analysts to lawyers, who want to interact with their content via a chat interface (the demo starts at 1:05:00).
Google describes Tailwind as an experiment that has been developed "over the past few weeks" by five Google professionals. In the U.S., if you are approved for Google Labs experiments, you can be placed on a waiting list for access.
Tailwind is interesting in that it should make it much easier in the future for individuals or companies to get access to custom language models that are fine-tuned to their personal data. OpenAI CEO Sam Altman has also announced that his company plans to offer customizable ChatGPT models in the future.
