Generative Pre-Training 2

Year: 2,019
Journal: OpenAI (Company)
Languages: English, French
Programming languages: Python
Input data:

Plain text

GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text. The diversity of the dataset causes this simple goal to contain naturally occurring demonstrations of many tasks across diverse domains. GPT-2 is a direct scale-up of GPT, with more than 10X the parameters and trained on more than 10X the amount of data.

Sign In

Register

Reset Password

Please enter your username or email address, you will receive a link to create a new password via email.