Meta’s AI lab has created a massive new language model that shares both
the remarkable abilities and the harmful flaws of OpenAI’s
pioneering neural network GPT-3. And in an unprecedented move for Big Tech, it is giving it away to researchers—together with
details about how it was built and trained.
...
Meta is making its model, called Open Pretrained Transformer (OPT), available for non-commercial use. It is also releasing its code and a logbook that documents the training process. The logbook contains daily updates from members of the team about the training data: how it was added to the model and when, what worked and what didn’t. In more than 100 pages of notes, the researchers log every bug, crash, and reboot in a three-month training process that ran nonstop from October 2021 to January 2022.
