Great set of research papers related to openly training large language models: https://openreview.net/group?id=aclweb.org/ACL/2022/Workshop/BigScience