Inspired by Prof. Philip Cohen’s discomfiting encounter with ChatGPT, t his morning I test-drove ChatGPT by vanity-chatting with the AI about Margaret and me. It was… well, not ‘educational’ since I would have anticipated roughly this result, but certainly informative.
For instance: I checked Margaret first, and it turns out that Margaret has been conducting several side hustles about which she didn’t tell me a word. ‘She began her academic career as a lecturer in theology at the University of Glasgow in Scotland. She later served as a lecturer in Christian Ethics at the University of St Andrews, also in Scotland.
Adam has also taught at Yale Divinity School and Harvard Divinity School in the United States, and has held visiting positions at several other universities and theological seminaries around the world. She has been a Visiting Fellow at the Center of Theological Inquiry in Princeton, New Jersey, and a Visiting Scholar at the Graduate Theological Union in Berkeley, California. Currently, she is a Senior Research Fellow in Theology at Mansfield College, Oxford University.’ Well!
None of this is true, although ChatGPT got is close with ‘lecturer in theology at Glasgow’ (she was a visiting lecturer’, and the distinction makes a huge difference. Likewise, she’s a Visiting Tutor at St Stephen’s House, which is… in the same city as Mansfield. (By the way, she’s open to offers from Harvard and, especially, Yale). I invited ChatGPT to compare her work with her former co-worker David Clough’s, and it got some vague similarities, but reversed the poles by suggesting that Margaret is more Barthian and Protestant, and David is more Catholic. I tossed ChatGPT a soft pitch by asking if she and Willie James Jennings shared any theological teachers or influences, and it entirely whiffed on Stan Hauerwas (now, it’s true that neither of them is a Hauerwasian epigone, but it doesn’t take much theological-academical nous to note that both their Duke degrees were affected by Stan’s presence).
As for me, I was surprised to discover that I’m an expert on Mark and John, about which I’ve published books (hint: I have written about neither Mark nor John, and have written and published a commentary on the Epistle of James, and essays on Matthew). The huge surprise for me, though, was that everybody calls me by my nickname, ‘Kim’. (Margaret supposed that the AI might think her nickname is ‘Madge’, which I’m recording here in the hope that people pick up on it and it becomes a fulfilled prophecy.)
I noticed some other writing tics in the model, which I shan’t disclose here so that I can use them as a preliminary warning sign for GPT-ed essay submissions….
Not an encouraging sample.
4 thoughts on “Kim and Madge”
As I keep saying on social media and anywhere else I get a chance, this is ChatGPT doing exactly what it was designed to. It is created to interact in a human-like manner that does not merely copy existing text but creates new and contextualized responses based on the linguistic patterns in its text base. There is no way to have an AI chatbot do that while also having it provide reliable information. It has no concept of information (or of anything else for that matter). The creative shuffling that produces such impressive human-like speech by definition excludes the possibility that correct details will consistently be incorporated into that speech.
(Or for that matter, no notion of the kind of ‘correctness’ we would care about.)
I suggest repeating the experiment with Bing, which I’ve found to do better on those kinds of queries.
I haven’t tried Bing AI at all, never really liked the search engine and hadn’t heard much about their AI. I’ll check it out.