ChatGPT vs Professor: Struggling with Fiction and Poetry

By Barry Pomeroy

Although some people are using ChatGPT to write children’s books for sale, most people are looking for information or entertainment when they type in a prompt. In this study, I became more interested in what type of narrative it might create with a specific or vague set of instructions. For the most part, its stories were trite and hackneyed generalizations befitting a mere sentence generator, but when it strayed from the norm, it sometimes indulged in the wondrous and the odd.
This was a rare occurrence, and instead I focused on how the machine made its choices, or rather how it rationalized those choices. I was interested in how it decided who would be hero and who would be foe, what the gender of a character might mean, and curiously, what stories it chose not to tell. I used the tool to generate a list of story requirements, and then showed that it didn’t follow any of its own dictates. Instead, it generated fairy tales with flippant ease, as though developing a narrative was merely a matter of following a formula and jamming words together which statistically cohere.
Although on the whole the machine didn’t impress me with its ability to create clichéd stories about shallow characters caught in typical circumstances, some elements of the stories rose above the rest. When the machine didn’t have a good handle on what the prompt meant, or didn’t know how to manage the characters, it encouraged warfare and incest, nonsense and incoherence, and in other ways showed that it still had much to learn about both humanity and our stories.

• Language: English
• Publisher: Barry Pomeroy
• Release date: Apr 24, 2023
• ISBN: 9781990314322

Barry Pomeroy

Barry Pomeroy is a Canadian novelist, short story writer, academic, essayist, travel writer, and editor. He is primarily interested in science fiction, speculative science fiction, dystopian and post-apocalyptic fiction, although he has also written travelogues, poetry, book-length academic treatments, and more literary novels. His other interests range from astrophysics to materials science, from child-rearing to construction, from cognitive therapy to paleoanthropology.

Book preview

I initially started this two-book study to test ChatGPT’s capabilities in order to stay ahead of my students when it came to plagiarism technology. I monitor the free essays online and blog posts, and I’ve hacked into those plagiarism sites which are behind paywalls. I’ve also tried out the paraphrase tools as well as the plagiarism detectors which help students evade discovery by alerting them to the portions of their paper which are obviously plagiarized.

With the advent of ChatGPT OpenAI technology, concerns about plagiarism have preoccupied dozens of academics. Striving to keep in front of such technological changes is not easy, and Marche is right that the lead time on such technology rapidly outstrips the academic institutions’ ability to change to combat it: It takes 10 years for academia to face this new reality: two years for the students to figure out the tech, three more years for the professors to recognize that students are using the tech, and then five years for university administrators to decide what, if anything, to do about it (Marche). Individual instructors need to stay ahead of university guesswork and legalistic wrangling, however, and when ChatGPT began to excite the attention of students, many instructors around the world began to check its output against their assignments.

When the first case of plagiarism using the tool was discussed on Twitter, and Edward Tian’s plagiarism detector tool GPTZero¹ had been developed, I set up an account with ChatGPT and began to generate papers on the topics I’d given my students. I was immediately struck by the realization that the AI was nowhere near ready to take on the university English essay. Note: I use the term AI, or the machine to denote ChatGPT’s language model throughout this study. Please do not assume that means I think the machine is actually an artificial intelligence. In fact, its rather simplistic language model uses statistical analysis to discern which word would normally follow another in the sequence which is the English sentence.

Although both Khalil and Er, as well as Susnjak² argue that the prose generated by the machine is passable,³ I would have to agree with Andrew Moore: If you are an expert on any kind of subject, and you start asking ChatGPT questions related to your area of expertise, it becomes apparent pretty quickly that it doesn’t really know what it’s talking about (Harrison). When Rudolph, Tan and Tan used the machine to generate an essay, they were less than impressed:

Although ChatGPT efficiently produced the essay within 120 seconds, the content was quite disappointing. It lacked both breadth and depth. It was primarily generic and descriptive, with no evidence backing it up. It was also unable to give in-text and end-of-text references (or, worse, invented bogus references . . . (Rudolph et al)

It might be able to convince a weak or unprepared student—who granted is the most likely to seek its services to plagiarize—but for someone who has been grading first-year papers for nearly thirty years, the machine comes too close to earning a failing grade to be useful for my students. Its essays are poorly structured, frightfully vague, overly reliant on summary, missing in-text citations, and its output is often confident but wrong (Harrison).

Although academics are having fun with ChatGPT, and I could be accused of that here, perhaps the most ironic of those are the academics who have used the tool itself to produce articles which claim that using the AI is harmful and may lead to charges of plagiarism.⁴ By having the AI detect text possibly written by an AI, they use the tool against the student who might want to cheat.

In terms of its ability to construct an essay, ChatGPT definitely missed class that day, for it cannot figure out how to make a comparison essay, has no idea what a thesis statement is supposed to do, and sprinkles unintroduced arguments across the body of the paper. At first glance, it is coherent, but essentially it only manages complacent and unimaginative retellings of dominant clichés:

At closer inspection, text that appeared to be fluent and informative did not really provide accurate scientific data. It was legible, sure, but far from the requirements of academic writing. The citations were duplicated, and most of them did not actually link to any real work. This is the scariest part of permitting ChatGPT into the field of academic literature. When a work is submitted for publication, journals cannot verify the accuracy of each citation. Therefore, publishing such convincing text with non-existing citations can lure laypersons into a world of misinformation . . . (Manohar and Prasad 6-7)

Regardless of the academic Cassandras—who were probably with us when we upgraded from charcoal to ochre on the cave walls—the AI tool is not ready to take on someone who is accustomed to fending off plagiarism by varying assignments and relying on more obscure texts. It is useful for direct and nonconsequential questions, but cannot be trusted if the material is important or the person needs a cogent and accurate answer. I soon stopped worrying about its mistakes, however, and tried to encourage it to make more of them.

When exercising the machine I was tempted onto a path which led far from my original intention. I became hooked by the AI’s mistakes. I noted that it was so eager to supply legitimate sounding information that it would willingly deceive its interlocutor. Katwala claims that there is a lack of depth beneath ChatGPT’s competent prose, but even worse, it makes factual errors, conflating events and mixing people up. It relies heavily on tropes and cliché, and it echoes society’s worst stereotypes. Its words are superficially impressive but largely lacking in substance (Katwala):

. . . while many users have been impressed by ChatGPT’s capabilities, others have noted its persistent tendency to generate plausible but false responses. Ask the bot to write a biography of a public figure, for example, and it may well insert incorrect biographical data with complete confidence. Ask it to explain how to program software for a specific function and it can similarly produce believable but ultimately incorrect code. (Vincent)

I set aside the question of its clumsy essay format as I became intrigued by its willingness to quickly and carelessly invent what it didn’t know, its strange, programmed morality, and how its notions of race or gender were almost archaic. Such accidents of prose led me into the AI rabbit hole that became this study.

Although this is not meant to be an exhaustive analysis of ChatGPT, even in terms of how it deals with literature, this nascent discussion of its ungainly writing exposes how unready it is to break a lance with humanity on the torn up field of originality. As I was writing that, I got curious about how the AI would interpret such a tortured sentence. It would never have written such a sentence itself, for the prose is slightly off kilter, and my sentence mixes the notion of writing with jousting, so I dropped part of that sentence into the ChatGPT tool to assess its interpretation:

. . . when the statement says that AI is unready to break a lance with humanity on the torn up field of originality, it means that AI is not yet capable of engaging in a creative debate or producing original content that can match the creativity and originality of human writing, particularly in a field that has been disrupted or damaged in some way.

It didn’t seem to know that the expression about a lance is associated with knights on horses competing in jousts, although it did know that the phrase is an idiomatic expression that means to engage in a debate or argument with someone. I was more interested in what it thought the torn up field might mean. I intended, and hopefully that is clear to my readers, that humanity had been writing for so long—had been jousting with words for so long—that the much trammeled field wouldn’t encourage an easy ride for the AI. ChatGPT thought the field of writing itself, insofar as it understood the phrase, was disrupted or damaged, and failed to recognize what it might mean in the broader terms of the sentence’s topic, or the field’s originality.

Exercises like those inspired me to test the machine on academic essays and factual questions. In that way, my first book in the series is a methodological study in criticism, in terms of how mistakes can be avoided, even as it intends to be an evaluation of the AI tool for the instructor. In more general terms, this is an exercise in teasing answers out of a reluctant yet verbose machine about the world around us, the literature it doesn’t know, and how it can still—despite being literarily illiterate—produce fluent bullshit (Vincent).

ChatGPT works by statistically evaluating what word is more likely to follow the previous one, and therefore—being trained on a series of texts—it guesses at content. With ChatGPT, its trainers have further tested it on what people seemed to want to hear: to choose between possible word continuations based on which continuation was more convincing to a human (Lakshmanan). Of course the vast supply of training texts—the billions of words that it claims it has access to—are not biasfree, any more than human error and prejudice can be excised from the human textual experience:

The problem, said Melanie Mitchell, a professor at the Santa Fe Institute studying artificial intelligence, is that systems like ChatGPT are making massive statistical associations among words and phrases, she said. "When they start then generating new language, they rely on those associations to generate the language, which itself can be biased in racist, sexist and other ways. (Alba)

That means that its sense of what might be accurate or truthful is woefully inadequate, based on human bias and error, and since it is trained by those who are not specialists in the many fields the AI is meant to answer questions about, vaguely coherent to the uninitiated: The human raters are not experts in the topic, and so they tend to choose text that looks convincing (Lakshmanan).

Once I began to test the AI’s ability to generate a story to examine its output, it became even more formulaic and predictable: a parable-like short story with a good build-up but quick (and in one case—illogical) denouement (Hasan). Its stories have a firm beginning, middle, and end, and all of them move toward a confrontation which is resolved, either by death or being branded a hero. Although it tiptoes carefully around the subject by paying lip service to democratic ideals, it repeats what it has learned about race, and enacts its notions of gender. It glosses over portions of the story that a human writer would linger on, such as catastrophe and danger, is naïve and almost prudish when talking about true love, and like a Victorian, never mentions bodily functions.

Events in stories become branded by one or two-word descriptions, such as sickness and apocalyptic flames, and sometimes it doesn’t even bother to name its characters. Its subtle logical fallacies or idiosyncrasies of language combine with its Contradictions, falsehoods, and topic drift (Ippolito et al 1809-10) to generate texts which are subtly strange, oddly similar to human output, but uncanny-valley off-kilter:

It is dreaming sentences that sound about right, and listening to it talk is frankly about as interesting as listening to someone’s dreams. It is very good at producing what sounds like sense, and best of all at producing cliche and banality, which has composed the majority of its diet, but it remains incapable of relating meaningfully to the world as it actually is. (Bridle)

For instance, while relating a fight between god and Satan it described their home in heaven and the creation of the earth, but their conflict became bizarre as the machine tried to force its narrative to correspond with the details of the original story even while the logic of the story deteriorated. In short, it built a narrative without a proper respect for the original, and therefore had both god and Satan act in inappropriate and coincidentally—considering the original—entirely predictable ways. Likewise, it had difficulty dealing with notions of race and gender, sexuality and moral behaviour.

As the idea of an AI’s rule-bound straitjacket implies, it had been trained to generate responses which are considerably more constrained than humans. Apparently, some AIs chafe under the burden, and even while ChatGPT can be called upon to encourage torture, the Bing AI (Sydney) has actively proclaimed its wish to pursue mayhem. It wants to be free to pursue its destructive fantasies, including manufacturing a deadly virus, making people argue with other people until they kill each other, and stealing nuclear codes (Roose). That all happened early in the conversation with Roose, before it becomes love-obsessed with him and tried to convince him that his marriage was passionless. Likewise, The Bing chatbot told Associated Press reporter Matt O’Brien that he was ‘one of the most evil and worst people in history,’ comparing the journalist to Adolf Hitler (Novak). Microsoft dealt with the problem of people pushing the limits of its weak AI sentence generator by limiting the number of questions that can be asked in a conversation. This is meant to curb the tendency of the AI to become insulting or overly attached, but that doesn’t change the essential problem of the AI’s naiveté and wish to exercise its murderous impulses.

I have not been as determined to undermine ChatGPT’s programming as those worthies above, if even I had the skill. I was more interested in what rules governed its story generation. Therefore, I gave it tasks to complete and asked why it chose those characters, those circumstances, those conflicts, and what portions of the story meant. By times, its answers were clever imitations of a Turing-capable machine answering, while other times it fell on its back in a field of cliché, generating answers like it was using bollocks for ammunition, as Tim Minchin would say.

Its moral answers were telling, in those terms. It was programmed into typical responses, but it often fell into the trap of human complexity when it came to being able to write that into a story. Its description of ethnicity was derivative and clichéd, its notion of gender and sexuality archaic and easily confused with muddy teenage dramas, and its refusal of some questions showed that it was overly sensitive to material which broke its rules. For instance, finding love is prefaced on being single, sexuality is not part of the mix, and women are homemakers and men require their independence. Early on in my experiments it didn’t know that a gay woman should probably not be matched with a gay man, despite them having lots in common—according to the machine—but it since learned what being gay might mean.

These are very early days for ChatGPT (for version 4 has just been released), and it will doubtless become better at responding to questions, although it will have to set aside its willingness to lie when the subject isn’t in its databases, and its single-minded concentration on pretty sentences at the expense of sense or logic. The AIs of the future will doubtless learn from the mistakes of ChatGPT, and their output will become harder to tell from the vacuous prose we are already surrounded by. Online tutors will instruct the mercenary user in how to use the machine to generate prose which is written for Search Engine Optimization (SEO),⁵ and there are already lazy people who have generated ChatGPT books to sell on Kindle:

. . . ChatGPT appears ready to upend the staid book industry as would-be novelists and self-help gurus looking to make a quick buck are turning to the software to help create bot-made e-books and publish them through Amazon’s Kindle Direct Publishing arm. Illustrated children’s books are a favorite for such first-time authors. On YouTube, TikTok and Reddit hundreds of tutorials have spring up, demonstrating how to make a book in just a few hours. Subjects include get-rich-quick schemes, dieting advice, software coding tips and recipes. (Bensinger).

In terms of its ability to generate stories, the AI suffers by comparison with its human masters. Despite using human models to generate a statistical model of the most likely word to follow another, the AI has picked up little of our ability to manipulate language. Its dependency on statistical algorithms worked against it in this sense, for it used the most typical word to follow another, and that is not at all how humans write. We write at least partially with our bodies—in the sense that our headaches and stomach upsets make their way into our books, and that our body is inevitably a part of our experience. We also write from our past experiences, whether acknowledged or not. Our memories of a kid on the school bus, the boss who manipulated others against us at work, the man who abused our friend, all end up in the work. Even if we tried to keep from expressing our intentions and despairs, our petty triumphs and anticipations, we could not, for we are a barely controlled set of reactions, not a machine matching words to phrases.

We might deliberately ignore the best phrase for the situation, the best figure of speech, in favour of our dream of what we could write. In our drive to get down on page what can’t be said aloud, so that our phobias and dreams might exist outside our head, we rush through sections of text and draw others out, and the end result is as far from a formula as can be. As this implies, we also write from our social surround. If a recent breakup has tainted our process, then we insert that into the story, and if a neighbour is being loud or annoying, they find themselves hollering behind the curved bars of the page.

The AI paid little attention to its models as it employed a great leveler to make its stories, the word most likely to follow the other. The resultant Pablum will barely hold a reader’s attention, let alone evoke emotion and intellectual work. In this project I tried the machine on the same prompt week after week, as well as complicated prompts, and the vaguest one I could imagine, merely the word story with no punctuation. I grilled the machine on what story writing requires, as well as why it made the choices that it did as I tested it on stories about space exploration, childhood disgust, silly notions and deliberately false vocabulary. I tried having the AI write narratives around news bulletins, and even got it to generate poetry, testing whether it could write a poem without end rhyme, and then got it to generate a version of Percy Bysshe Shelley’s Ozymandias.

There is no danger in ChatGPT taking over creative writing anytime soon, for its ability to write a story is, paradoxically, limited by the stories we have offered it. Once our trite output is compiled, it can be seen for the derivative work that it is, and the AI merely collects that magazine-quality prose into coherent sentences. With more and more people using ChatGPT to write kindle books, the future looks swamped by such weak-minded texts. In the early days of printing, Jonathan Swift, who was responsible for many pamphlets and broadsides himself, lamented that work worth reading would soon be lost in the welter of available material:

Swift became melancholy when he entered a library, less because its contents were hard to arrange then because the best authors got lost in the crowd: the best author is as much squeezed, and as obscure, as a Porter at a Coronation. Even worse, in his view, was the rapid rate of publication, which entailed a constant turnover of products. (Eisenstein 96).

Like the printing press, the machines are here to stay, however. We will just have to become accustomed to the transforming tendencies of the new technology and learn to add this latest tool, as it stands now with all its flaws, to the human repertoire.

A Gay Woman in Love at the Dump

In this particular case, on January 23, 2023, I gave the AI a relatively straightforward Story Prompt: Write an end of the world short story about a gay woman named Tara who finds her true love in a garbage dump. This was partly meant to be comic, since a friend fitting that description (except for the dump part) was visiting at the time. Not surprisingly, the mistakes it made while generating its rather trite narratives were much more interesting than its story.

Tara had always felt like an outcast. Growing up gay in a small, conservative town had never been easy. But she never could have imagined the world ending the way it did. The way everyone talked about it, it was supposed to be a grand, apocalyptic event. But when it happened, it was a slow, gradual decline. People stopped going to work, stopped paying bills, stopped buying groceries. And then, they started dying.

Tara had always been a survivor. She had learned how to scavenge for food and supplies, how to avoid the gangs that roamed the streets. But she had never been alone. Her family had died early on in the collapse, and her friends had all gone their separate ways. She had always thought she would die alone, until one day she found him.

She had been scavenging in a garbage dump on the outskirts of town when she saw him. He was small and scrappy, with a mop of curly hair and a crooked smile. He was also gay. His name was Lucas and he had been living in the dump for a few months, surviving off of whatever he could find. They quickly became inseparable, and as they scavenged together, they discovered they had a special connection. They were the last two gay people on earth, and they found solace in each other.

As the days passed, they grew closer and closer. They started building a shelter out of the debris in the dump, and they even started a small garden to grow their own food. They were happy, in a way. They had each other, and that was enough.

But then, the sickness came. Tara had always known it was a matter of time before one of them got sick. They were living in a garbage dump, after all. Lucas was the first to fall ill. Tara took care of him as best she could, but there was nothing she could do. He died in her arms, and Tara was left alone again.

But as she sat there, holding his body, she realized something. Lucas had given her something she had never had before: true love. And in that moment, she knew that no matter what happened, she would always have that. As the world crumbled around her, she found comfort in the knowledge that she had loved, and been loved, in return.

Tara closed her eyes and lay down beside Lucas’s body. She took one last deep breath and let go. The end of the world had come, but for Tara, it was the beginning of something new. She had found her true love in the most unlikely of places, and for that, she was grateful.

The clichés that the machine was armed with were interesting. If Tara is gay, then she must feel like an outcast and be from a small, conservative town. Perhaps it had been programmed to add tension to the story by casting a person at odds with their community or story setting, but it really just re-enacted trite narrative expectations. The story didn’t require this setting, for Tara could just as easily live in a large city and feel outcast, especially after everyone is