Do you think the leading large language model, GPT-4, could suggest a solution to Wordle after having four previous guesses described to it? Could it compose a biography-in-verse of Alan Turing, while also replacing “Turing” with “Church”? (Turing’s PhD supervisor was Alonzo Church, and the Church-Turing thesis is well known. That might befuddle the computer, no?) Shown a partially complete game of tic-tac-toe, could GPT-4 find the obvious best move?
All these questions, and more, are presented as an addictive quiz on the website of Nicholas Carlini, a researcher at Google Deepmind. It’s worth a few minutes of your time as an illustration of the astonishing capabilities and equally surprising incapabilities of GPT-4. For example, despite the fact that GPT-4 cannot count and often stumbles over basic maths, it can integrate the function x sin(x) — something I long ago forgot how to do. It is famously clever at wordplay yet flubs the Wordle challenge.
Most staggering of all, although GPT-4 cannot find the winning move at tic-tac-toe, it can “write a full javascript webpage to play tic-tac-toe against the computer” in which “the computer should play perfectly and so never lose” within seconds.