Talking with may feel like a conversation with a friend, but it functions as a signed confession. The Washington Post’s Miriam Waldvogel and Gerrit De Vynck report on the evidentiary role that AI models are playing in court. I don’t mean using AI to file court cases (sometimes with precedents the AI fabricated), but treating AI — or rather the chat transcripts — as evidence. My colleague Alana covered a story like this before, but what makes this reportage different is that it is comprehensive: twelve court cases, both civil and criminal, over the past two years. (One caveat: AI chat transcripts cited in court cases are a subset of AI chat transcripts used in all legal proceedings. If a chat transcript came up in closed-door negotiations that led to a private settlement, then of course we’d have no idea.)
One case to illustrate: Ryan Schaefer, a student at Missouri State, smashes seventeen cars (not with his own car by accident, but as vandalism), then asks ChatGPT, “Is there any way they could know it was me?” Since the conversation also includes such highlights as, “I was smashing the windshields of random fs cars,” there really isn’t any ambiguity about what “it” might refer to here. It isn’t clear to me whether Schaefer would have still been convicted without the transcripts. He handed over his phone after he was made a suspect but before he was arrested. In any case, though, he opened his phone to the police, they read the transcript, and he has since been sentenced to five years’ probation.
Part of the problem has nothing to do with AI models. In several of the cases mentioned, chat transcripts were usable at all because somebody voluntarily handed over their device and opened it up to investigators. (This is what the experts call a “bad idea.”) Another part is pure technological incompetence: Today, people are leaving evidence in AI conversation transcripts, but yesterday they were Googling “how do I clean bloodstains” and leaving incriminating texts on their phone.
But why is this happening now? I mean, The Washington Post published an article about this now, and not five years ago. The facile answer is that modern AI systems didn’t exist then: maybe somebody asked GPT-3 for legal advice, but that wasn’t a widespread phenomenon. But I think that increased capabilities are another reason: it’s becoming easier to mistake AI models as people. They give useful advice and maybe they can be trustworthy, nonjudgmental friends — but the AI model is not your confidant and cannot be so. It may feel like a conversation with a friend, but it functions as a signed confession.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
You can receive emails of dispatches as we write them, or subscribe to our Daily Digest for a once-a-day compilation.