From accusations that led Harvard University’s president to resign in January, to revelations in February of plagiarized text in peer-review reports, the academic world has been roiled by cases of plagiarism this year.

But a bigger problem looms in scholarly writing. The rapid uptake of generative artificial intelligence (AI) tools — which create text in response to prompts — has raised questions about whether this constitutes plagiarism and under what circumstances it should be allowed. “There’s a whole spectrum of AI use, from completely human-written to completely AI-written — and in the middle, there’s this vast wasteland of confusion,” says Jonathan Bailey, a copyright and plagiarism consultant based in New Orleans, Louisiana.

Generative AI tools such as ChatGPT, which are based on algorithms known as large language models (LLMs), can save time, improve clarity and reduce language barriers. Many researchers now argue that they are permissible in some circumstances and that their use should be fully disclosed.

But such tools complicate an already fraught debate around the improper use of others’ work. LLMs are trained to generate text by digesting vast amounts of previously published writing. As a result, their use could result in something akin to plagiarism — if a researcher passes off the work of a machine as their own, for instance, or if a machine generates text that is very close to a person’s work without attributing the source. The tools can also be used to disguise deliberately plagiarized text, and any use of them is hard to spot. “Defining what we actually mean by academic dishonesty or plagiarism, and where the boundaries are, is going to be very, very difficult,” says Pete Cotton, an ecologist at the University of Plymouth, UK.

In a 2023 survey of 1,600 researchers, 68% of respondents said that AI will make plagiarism easier and harder to detect. “Everybody’s worried about everybody else using these systems, and they’re worried about themselves not using them when they should,” says Debora Weber-Wulff, a plagiarism specialist at the University of Applied Sciences Berlin. “Everybody’s kind of in a tizzy about this.”

Plagiarism meets AI

Plagiarism, which the US Office of Research Integrity defines as “the appropriation of another person’s ideas, processes, results, or words without giving appropriate credit”, is a familiar problem. A 2015 study estimated that 1.7% of scientists had admitted to plagiarism and that 30% knew colleagues who had committed it¹.

LLMs could make things worse. Intentional plagiarism of human-written text can easily be disguised if someone asks an LLM to paraphrase the wording first. The tools can be prompted to paraphrase in sophisticated ways, such as in the style of an academic journal, says Muhammad Abdul-Mageed, a computer scientist and linguist at the University of British Columbia in Vancouver, Canada.

A central question is whether using unattributed content written entirely by a machine — rather than by a human — counts as plagiarism. Not necessarily, say many researchers. For example, the European Network for Academic Integrity, which includes universities and individuals, defines the prohibited or undeclared use of AI tools for writing as “unauthorized content generation” rather than as plagiarism as such². “Plagiarism, for me, would have things that are attributable to another, identifiable person,” says Weber-Wulff. Although there have been instances of generative AI producing text that looks almost identical to existing, human-written content, it is usually not close enough to be considered plagiarism, she adds.

Plagiarism in peer-review reports could be the ‘tip of the iceberg’

However, some people argue that generative AI tools are infringing copyright. Both plagiarism and copyright infringement are the improper use of someone else’s work, and whereas plagiarism is a breach of academic ethics, unauthorized use of copyrighted work can be a breach of the law. “These AI systems are built on the work of millions or hundreds of millions of people,” says Rada Mihalcea, a computer scientist at the University of Michigan in Ann Arbor.

Some media companies and authors have protested against what they see as copyright breaches by AI. In December 2023, The New York Times launched a copyright lawsuit against the tech giant Microsoft and OpenAI, the US company behind the LLM GPT-4, which powers the chatbot ChatGPT. The lawsuit claims that the two firms copied and used millions of the newspaper’s articles to train LLMs, which now compete with the publication’s content. The lawsuit includes instances in which prompts caused GPT-4 to reproduce several paragraphs of the newspaper’s articles almost word for word.

In February, OpenAI filed a motion in federal court to dismiss parts of the lawsuit, arguing that “ChatGPT is not in any way a substitute for a subscription” to The New York Times. A spokesperson for Microsoft says that “lawfully developed AI-powered tools should be allowed to advance responsibly”, and “they are also not a substitute for the vital role that journalists play”.

If a court rules that training an AI on text without permission is indeed copyright infringement, “that’s going to be a huge shake up for AI companies”, says Bailey. Without extensive training sets, tools such as ChatGPT “can’t exist”, he says.

AI explosion

Whether it’s called plagiarism or not, the use of AI in academic writing has exploded since ChatGPT was released in November 2022.

In a preprint updated in July³, researchers estimated that at least 10% of abstracts in biomedical papers in the first six months of 2024 had used LLMs for writing — equivalent to 150,000 papers per year. The authors, led by data scientist Dmitry Kobak at the University of Tübingen in Germany, analysed 14 million abstracts in the academic database PubMed that had been published between 2010 and June 2024. They showed that the arrival of LLMs was associated with the increased use of stylistic words — such as ‘delves’, ‘showcasing’ and ‘underscores’ — and then used these unusual word patterns to estimate the proportion of abstracts that had been processed using AI (see ‘AI in academic papers’). “The appearance of LLM-based writing assistants has had an unprecedented impact in the scientific literature,” they wrote.

Kobak and his colleagues found that papers from countries including China and South Korea showed signs of heavier LLM use than did those from countries where English is the dominant language. However, says Kobak, authors in this latter group of countries might be using these tools just as often, but in ways that are more difficult to spot. Use of LLMs “will certainly continue to increase”, Kobak predicts, and will “probably get harder to detect”.

The undisclosed use of software in academic writing is not new. Since 2015, Guillaume Cabanac, a computer scientist at the University of Toulouse, France, and his colleagues have been uncovering gibberish papers made by software called SCIgen, and ones containing ‘tortured phrases’ that were created by automated software that translates or paraphrases text. “Even before generative AI, people had tools to fly under the radar,” Cabanac says.

And some use of AI in academic writing has value. Researchers say that it can make text and concepts clearer, reduce language barriers and free up time for experiments and thought. Hend Al-Khalifa, an information-technology researcher at King Saud University in Riyadh, says that before generative AI tools became available, many of her colleagues for whom English is a second language would struggle to write papers. “Now, they are focusing on the research and removing the hassle of writing with these tools,” she says.

But confusion reigns about when the use of AI constitutes plagiarism or contravenes ethics. Soheil Feizi, a computer scientist at the University of Maryland, College Park, says that using LLMs to paraphrase content from existing papers is clearly plagiarism. But using an LLM to help express ideas — either by generating text based on a detailed prompt, or by editing a draft — should not be penalized if it is done transparently. “We should allow people to leverage large language models to be able to express their ideas effortlessly and in a clearer manner,” Feizi says.

Many journals now have policies that allow some level of LLM use. After originally banning text generated by ChatGPT, Science updated its policy in November 2023 to say that use of AI technologies in writing a manuscript should be disclosed in full — including the system and prompts used. Authors are accountable for accuracy and “ensuring that there is no plagiarism”, it says. Nature, too, says authors of research manuscripts should use the methods section to document any LLM use. (Nature’s news and features team is editorially independent of its journals team.)

‘Obviously ChatGPT’ — how reviewers accused me of scientific fraud

An analysis of 100 large academic publishers and 100 highly ranked journals found that by October 2023, 24% of publishers and 87% of journals had guidelines on the use of generative AI⁴. Almost all of those that provided guidance said that an AI tool could not be included as an author, but policies differed on the types of AI use allowed and the level of disclosure required. Clearer guidelines on AI use in academic writing are urgently needed, says Weber-Wulff.

For now, the rampant use of LLMs for writing scientific papers is curbed by their limitations, says Abdul-Mageed. Users need to create detailed prompts describing the audience, the style of language and the subfield of research. “It’s actually very difficult for a language model to give you exactly what you want,” he says.

But developers are building applications that will make it easier for researchers to generate specialized scientific content, says Abdul-Mageed. Rather than having to write a detailed prompt, a user could in future simply pick from a drop-down menu of options and push a button to produce an entire paper from scratch, he says.

Detective work

The rapid adoption of LLMs to write text has been accompanied by a flurry of tools that aim to detect it. Although many boast high rates of accuracy — more than 90%, in some cases — research has suggested that most do not live up to their claims. In a study published last December⁵, Weber-Wulff and her colleagues assessed 14 AI-detection tools that are widely used in academia. Only 5 accurately identified 70% or more of texts as AI- or human-written, and none scored above 80%.

The detectors’ accuracy dropped below 50%, on average, when spotting AI-generated text that someone had lightly edited by replacing synonyms and reordering sentences. Such text is “almost undetectable by current tools”, the authors wrote. Other studies have shown that asking an AI to paraphrase text multiple times drastically reduces the accuracy of the detectors⁶.

AI and science: what 1,600 researchers think

There are other problems with AI detectors. One study showed that they are more likely to misclassify English writing as AI-generated if it was penned by people for whom English is not a first language⁷. Feizi says the detectors cannot reliably distinguish between text written entirely by AI and cases in which an author used AI-based services that polish text by helping with grammar and sentence clarity. “Differentiating between these cases would be quite difficult and unreliable — and could lead to a huge rate of false positives,” he says. Being falsely accused of using AI, he adds, can be “quite damaging to the reputation of those scholars or students”.

The boundary between legitimate and illegitimate use of AI is likely to blur further. In March 2023, Microsoft started to incorporate generative AI tools into its applications, including Word, PowerPoint and Outlook. Some versions of its AI assistant, called Copilot, can draft or edit content. In June, Google also began integrating its generative AI model, Gemini, into tools such as Docs and Gmail.

“AI is becoming so embedded in everything we use, I think it’ll become increasingly difficult to know whether something you’ve done has been influenced by AI,” says Debby Cotton, a specialist in higher education at Plymouth Marjon University, UK. “I think it’ll carry on evolving more rapidly than we can keep pace with.”