Dispatches from Mitch
Did another short story prize just go to AI?
If it walks like a duck and quacks like a duck, it’s probably a duck. And if it sounds like an AI-written short story and gets a score of “100% AI Generated” from Pangram, it was probably written by an AI, even if you might never know for sure.
The latest prizewinner to come quacking this way is “Back and Forth”, a short story by (ostensibly) Kavyta Kay that was submitted to the Harper’s Bazaar Short Story Competition. Published Thursday by the magazine, the story was said to be judged by an “all-star” panel that includes the novelist Ruth Ozeki.
Alerting us to its more likely provenance is writer and researcher Nabeel S. Qureshi, who may have been the first to expose last month’s winner of the Commonwealth Short Story Prize as likely AI-written.
The latest news on the Commonwealth prize is that the literary magazine Granta has decided to stop publishing the winning entries from that contest, “for the sake of our own editorial integrity,” the magazine says.
On the new Harper’s story, Qureshi writes:
Literary prizes need to start including Pangram checks in their process, or else change the rules to make AI writing ok. It’s very simple!
Pangram isn’t perfect, but it is known for erring on the side of caution, where caution means maintaining a very low false positive rate. That makes the tool relatively easy to fool into thinking something mostly AI might be human written, but it also makes it pretty damning when anything longer than a couple of paragraphs comes back “100% AI Generated.”
I think AI educator and evaluator Ethan Mollick has the right explanation for why AI keeps winning contests:
AI is generally a weak fiction writer except for one particular kind of fiction (metaphor-rich, staccato sentences, short & plot light, etc.) which it writes excellently. This happens to be a style that can sometimes do quite well in modern literary fiction short story contests.
As for why AI is generally weak at fiction, the insider view is that fiction is both harder to train AI to do well and not a high training priority for AI companies. Training is most straightforward when there are clear objective metrics for success — like, “Does the code run?” or “Did I win the game?” — and storytelling is mostly not like that.
So why is it good at the short, poetic style? My guess would be that this is because, being short, it is the type of style that actually sometimes gets scored by humans somewhere in the training process. And humans are known for liking staccato punchiness and for being easily impressed by wild metaphors.
If it walks like a duck, a primordial duck, a duck lost to history and reclaimed by a bullied child with a missing tooth who needs it to be a puppy, then who is to say that it is not a puppy? Not his papá, lost in his bottle. Not his mamá, sleeping in the sky. Let the feathers fall in their iridescent season. Let barks be made of quacks.
Flat curve, meet exponential
The situation with Anthropic’s Fable and Mythos models has many asking if we’ve seen all the AI we’re going to see — if models past a certain threshold will never make it to a public release for fear of what they might do in the wrong hands.
Engineer and blogger Steve Yegge has made one of the fuller cases I’ve yet seen for this “flat curve” outcome. In Yegge’s forecast, AI keeps getting better, but only where you’ll never see it:
I am now in the camp who believe that we are only at most two or three model generations away from AI finally being controlled like nuclear weapons. Only a few will have access to superintelligence above the classes of models we’re seeing this year.
With my caveat that if anyone builds something worthy of the name “superintelligence” we’re probably all dead shortly after, I agree it’s possible that governments will clamp down, “with the chokepoint being the supply chain.”
I also agree with Yegge that for every user there is a threshold of AI capabilities beyond which said user is unable to judge the quality of the AI’s outputs. As he puts it, “Superhuman means unverifiable.”
Where I part ways is when he takes this fact to mean that companies will have no need of an AI that “produces work that nobody can grade.” He calls acquiring such AI “A terrible outcome, assuming you don’t want to surrender your business to AI entirely.”
I part ways because I think many businesses will want to surrender to AI entirely, or will feel like they have no choice but to do so. At that point, I think humanity starts losing the planet for very mundane reasons familiar to anyone who has ever had any criticism of capitalism.
But perhaps governments would reach the same conclusion and disallow the training of superhuman models. In a world where consumer AI plateaus, we can consider Yegge’s other thoughts from this post. But they’re mostly business thoughts about software-as-a-service not going away after all, and about the kind of training workers will need to use AI agents effectively.
So let’s instead talk about what I think actually happens if, as Yegge seems to assume, the AI companies continue training better and better AIs that governments never let them release:
In reality, behind the scenes, the curve is NOT flattening at all; the exponential growth will continue, and you will be able to see outwardly observable signs of it, e.g. in data center growth.
If I entertain Yegge’s model that presumes we’re not all dead from this, then I notice that there must be someone paying for all that development. I doubt it could all come from a plateaued consumer market; this market would grow increasingly competitive and get gradually squeezed by open-weights models. So I’d assume there must be a business model where the next-gen, locked away AIs are doing a lot of useful work that investors expect consumers to pay for indirectly or eventually, like drug discovery, materials science, and robotics research.
With robotics especially, people outside the curtain might not notice anything was happening at all until it happened all at once — until robots built by robots built by robots were numerous enough that having them make more robots was no longer the most profitable thing for them to do and so were sent to do everything else instead.
Similarly, drugs take a long time to trial and approve, but there’s no hard cap on how fast they can be discovered. So after a long initial delay, the first blockbuster AI drug might be followed very closely by the second, the third, and the 30th.
In other words, I think the common argument that physical bottlenecks to rolling out AI tech will limit the rate of AI development is probably wrong. The world doesn’t have to adopt AI for AI to continue advancing behind closed doors — perhaps with the help of AI itself. If this happens, its economic impacts don’t disappear; they just quietly build up until they explode. Exponentials look like nothingburgers until they swallow you whole.
Yegge’s not making that bottleneck argument, and I bet he agrees with me about exponentials. But there is something odd about a post that points to an explosion happening behind a curtain and acts like it will stay there.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.