Gemini can scan your Google account to generate detailed replies in Gmail, using your data and writing style. These Personal Smart Replies worry me since they could distort human relationships and outsource authenticity.
As an AI advocate and enthusiast, I enjoyed Google I/O 2025. Gemini is now more helpful than ever, and Google introduced powerful new tools like Flow for quick and captivating video creation. In the midst of this excitement, Google demonstrated an AI feature that worried me — Gmail Personal Smart Replies.
Personal Smart Replies may be too convincing
I love AI if it's used appropriately, but when Google CEO Sundar Pichai shared a Personal Smart Replies example, it felt like a bad approach. After a friend emailed to ask for advice on a trip Sundar was familiar with, he let Gemini reply. The response was detailed and friendly but entirely fake.
Pichai sets up the example, saying, "Let's say my friend wrote to me looking for advice." It's about a trip to Utah similar to one Pichai made. In that context, a friend would expect a personal reply or a brief, standard automated message. Instead, Gemini scans Pichai's Google account, looking for relevant information in Photos, Drive, Gmail, and more.
Personal Smart Replies assembles a detailed response, perhaps too detailed. Pichai explains, "Gemini matches my typical greetings from past emails, captures my tone, style, and favorite word choices, and then it automatically generates a reply. I love how it included details like keeping driving time under five hours per day, and it uses my favorite adjective, exciting." To be fair, Pichai suggested you could make a couple of changes before hitting Send.
Still, I'm worried that this is the wrong direction for AI. I tend to use AI for research and image generation. I'm happy to let Gemini assist or even complete my work if the results are good enough. Fun images that add to a conversation or represent a concept add value. However, we shouldn't outsource our humanity by letting AI take over personal relationships.
How much AI is too much?
In the long term, I'd like to see AI and robotics free humans from the drudgery of work and make every job optional. Then we could focus on personal improvement, innovation, exploration, and creativity, instead of repetitive, menial work done simply to earn a paycheck.
Artists, writers, and creators of all kinds have expressed concerns about AI usage. Within the confines of capitalism, it's correct to question whether copyrighted material has been used to train AI models without compensation. However, I don't think AI tools will ever replace human creativity. True art can still convey visceral emotions and esoteric concepts that are foreign to AI models, even if they pretend to understand.
AI has the power to change our behavior. Since Personal Smart Replies only need a single click, without prompting, it's a tempting shortcut. However, removing all effort could create the false impression that you spent several minutes, even an hour, finding and reviewing your documents, maps, photos, and more. That's what Gemini does in seconds.
No matter how personal an AI-generated email reply sounds, it will always lack authenticity. On the other end of the conversation, the person who sent the original email might be impressed and think you want a deeper connection.
If it's someone you hardly know, the overly helpful reply could signal desperation or trying too hard. Imagine the confusion that could arise in a dating scenario. Singles will have to ask, "are they really that into me, or is that just an AI response?"
Finding the right balance
I believe the key to finding the right balance with Personal Smart Replies is imagining you're the recipient, lacking awareness that it's AI-generated. Does the email match how you'd actually respond? If you have any doubts about the depth or tone of the message, customize it.
Business emails are, by nature, somewhat facetious. There's a monetary incentive to maintain a friendly attitude, even if we're stressed or unhappy about an email. Business is about transactions, sometimes collaboration, but not a true friendship, though a closer connection may develop over time. Still, I might add a note for important contacts, expressing that I was quite busy, so I used AI to help generate a more detailed answer.
For friends and significant others, authenticity is crucial to good, long-lasting relationships. I want my loved ones to know and like me, not Gemini or any other overly helpful and friendly AI assistant. They need to learn my true style, including how long I'm willing to spend on a reply to their email, how conversational I am, and what topics I'm willing to discuss in depth. Otherwise, I'll seem like a different person when I'm physically present.
I like Personal Smart Replies
There's also something to be said for short emails that get right to the point. Someday, crafting a concise but friendly email could become a lost art. Perhaps there's no need for brevity if your AI summarizes every long-winded email my AI generates.
In some dystopian future, my AI could have a deep, ongoing relationship with your AI, while we never converse at all. That false friendship might last years, only to end abruptly when we meet in person and realize we're not as effusive and kind as our Personal Smart Replies suggest.
Despite that risk, I like the overall concept of Personal Smart Replies. Other AI models like Siri lack Gemini's deep integration. A helpful digital assistant that surfaces relevant data could help me share in-depth replies when appropriate, helping me express genuine interest in friends and sharing detailed facts with coworkers. Used wisely with some edits and possibly a brief note about AI assistance, it can improve communication without hurting authenticity.
