10 Facts About GPT-2

GPT-2 was created as a "direct scale-up" of OpenAI's 2018 GPT model, with a ten-fold increase in both its parameter count and the size of its training dataset.

GPT-2 was to be followed by the 175-billion-parameter GPT-3, revealed to the public in 2020 .

GPT-2 was created as a direct scale-up of GPT, with both its parameter count and dataset size increased by a factor of 10.

Notably, GPT-2 was evaluated on its performance on tasks in a zero-shot setting.

GPT-2's flexibility was described as "impressive" by The Verge; specifically, its ability to translate text between languages, summarize long articles, and answer trivia questions were noted.

Related searches

One commonly-cited justification was that, since generated text was usually completely novel, it could be used by spammers to evade automated filters; OpenAI demonstrated a version of GPT-2 fine-tuned to "generate infinite positive – or negative – reviews of products".

GPT-2 deployment is resource-intensive; the full version of the model is larger than five gigabytes, making it difficult to embed locally into applications, and consumes large amounts of RAM.

Possible applications of GPT-2 described by journalists included aiding humans in writing text like news articles.

In 2019, AI Dungeon was launched, which used GPT-2 to generate dynamic text adventures based on user input.

In February 2021, a crisis center for troubled teens announced that they would begin using a GPT-2-derived chatbot to help train counselors by allowing them to have conversations with simulated teens .