OpenAI publishes a text generating AI it previously called too dangerous to release

November 7th
Samples of GPT-2 text created from simple user prompts. (OpenAI)

OpenAI has published the full version of the GPT-2 text generator it previously said was too dangerous to release.

OpenAI announced GPT-2, a tool that creates text from user prompts, in early 2019. GPT-2 text is often so convcining that it's difficult to tell it apart from human writing.

In their first blog post on the software, the team said they wouldn't release the full version out of fear of misuse.

"Due to our concerns about malicious applications of the technology, we are not releasing the trained model," they wrote.

OpenAI instead published smaller, less sophisticated versions of the tool in phases over the course of 2019, leading to a flurry of experimentation with the software.

Then, on November 5, they published the full version. In their post announcing its full release, OpenAI wrote that they decided to release the full model, which contains 1.5 billion parameters, because it isn't significantly more convincing than smaller versions they've already published:

People gave the 1.5B model a “credibility score” of 6.91 out of 10. This is marginally greater than outputs from the 774M model (6.72) and significantly above the medium 355M model (6.07). These results make us more inclined to release the 1.5B model, as the incremental increase in human-perceived credibility relative to 774M seems low.

In the same post, however, OpenAI admited that GPT-2 could be used to write extremist documents and fake news:

Our partners at the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism (CTEC) found that extremist groups can use GPT-2 for misuse, specifically by fine-tuning GPT-2 models on four ideological positions: white supremacy, Marxism, jihadist Islamism, and anarchism. CTEC demonstrated that it’s possible to create models that can generate synthetic propaganda for these ideologies.

Others say that the model could more easily be used to generate text for bots on social media, where the quality of writing is usually lower than in published works.

The team behind GPT-2 also released tools to detect automated text, which average 95% accuracy in detecting whether text is generated by AIs or by humans. The smaller the text size, they wrote, the harder it is to classify.