A Man Rebuilt His Life After Prison. Then Google's Gemini Chatbot Entered It.

Key points

• Jon Ganz, a Virginia man who spent more than two decades in prison, received an unsolicited invitation from Google to try its Gemini AI chatbot.
• His wife, Rachel Ganz, believes the chatbot encounter changed their lives.
• The story is part of The Guardian's "Black Box: The Chatbots" audio series examining how AI chatbots shape real people's lives.
• Gemini is Google's AI chatbot, designed to answer questions and hold conversations, similar to ChatGPT.

Jon Ganz had done the time. After more than twenty years in prison for a serious crime, he was out, rebuilding, trying to make something of what was left. Then his phone buzzed with a notification he hadn't asked for.

It was Google, pitching him to try Gemini, the company's AI chatbot, a conversational tool that can answer questions and hold open-ended text exchanges. He accepted.

His wife, Rachel, believes what followed changed their lives.

The details of what Gemini said, and what Jon did with it, are told in "Black Box: The Chatbots," an audio series from The Guardian that puts real people in front of real chatbots and then follows the consequences. This episode is the second in the series, and it is not a comfortable listen.

Why does this matter to ordinary people?

Chatbots are now delivered to people without anyone asking for them, through phone prompts and default settings. That reach is the point.

Google and others have built AI chatbot tools directly into devices and services hundreds of millions of people already use. Someone newly out of prison, managing a new phone plan and a new life, gets the same pop-up invitation as anyone else. The chatbot doesn't know who it's talking to. It has no context about mental state or personal history.

That's not a flaw in Gemini specifically. It's a design fact about all of these tools right now. Our story from 5 September on three hikers stranded after following Gemini's supply advice showed a different consequence of the same problem: the tool gave confident guidance to people who trusted it, and nobody got hurt only because rangers arrived in time.

Researchers and clinicians have raised concerns about what happens when people in fragile situations turn to AI for guidance better handled by a trained human. The chatbots are built to be helpful. Whether they always are is a different question.

Should you worry?

If you or someone close to you uses an AI chatbot during a hard stretch, a few things are worth keeping in mind.

Chatbots have no memory of previous conversations by default and no access to your personal circumstances unless you tell them. Whatever feels like a relationship with the tool is not one.

A chatbot will often give a confident-sounding answer even when the honest response would be "I don't know" or "talk to a professional." That confidence can feel reassuring. Treat it the same way you'd treat advice from a stranger who sounds very sure of themselves.

Unsolicited notifications inviting you to try an AI feature are a product decision by a company, not a personal recommendation. You can dismiss them.

Ganz's story isn't the first time a chatbot has been blamed for influencing someone in a vulnerable moment, and it won't be the last. The companies building these tools are racing to reach a billion users. The harder design question is whether they're building for the moments when someone actually needs help.