Lately, I've been exploring the edges of their capacity, and have found areas of significant weakness...at least in the versions you don't have to pay big bucks to play with and preprogram.
It's the "P" in the GPT that seems to be the most fundamental weakness of these remarkable systems. If an AI is exploring an area that's known, and about which much has been written, they're able to engage accurately using the prior thoughts of humankind. ChatGPT 4 in particular has proven quite able on that front. As a pastor, I marvel at how competently it exegetes scripture, and is capable of nuanced discussion of matters of esoteric theology and church history.
But what about something that it hasn't ever encountered? How does it deal with the truly new?
To test that, I would need to ask it to explain texts that no human being has ever read, ones that have never been examined, interpreted, or reviewed. They would have no meaningful presence online. No preprogramming could possibly prepare an AI for the encounter.
But where to find such a text?
The literary implement I've used to completely deconstruct the illusion of GPT intelligence is a book. Not the Bible, which is rather well discussed at this point. Rather, the book that I've used is entitled A SLOW DEATH ON NEVSKY PROSPEKT. This magical, AI-befuddling book?
It's my self-published collection of literary sci fi short stories.
No-one's ever read these stories. No one's reviewed them, or commented on them. They're dang fine stories, in my humble opinion, but the publishing of short fiction being what it is, most have never seen the light of day.
They don't exist in the human knowledge base. They're narrative, utterly obscured by noise.
And that means ChatGPT 4.0 just can't handle them. It doesn't matter if I include a document as an attachment, or feed it to the AI in little bits in comments. Sometimes the AI presents a viable simulation of insight. Most times, it flounders and hallucinates with all of the same quasicoherence that ChatGPT 1.0 did. It gets trapped in feedback loops. It confuses one story for another. It shuts down, crashing regularly. It plain old just makes things up. It starts writing its own stories, and then won't talk about anything but those. It struggles like a kindergartener asked to write an essay on Anna Karenina.
Some of that likely stems from the limitations of the tokenization process, as it breaks down words into fundamental units. There are just too many words, too many concepts. But again, I suspect that it's primarily that it can't lean on preprogramming at all. Not at all. None of the words it needs already exist.
Over the last week or two of exploration, the illusion of LLM near-sentience has completely collapsed. ChatGPT 4.0 is clearly not aware, nor is it intelligent. It can't handle the encounter with the new. It feels as far from sentience as a Loebner Prize runner up in 2010.
One day, perhaps. But that day is not today.
Posts
