GPT-NL: Developing a Dutch LLM from scratch - Julio A. de Oliveira Filho, Athanasios Trantas скачать в хорошем качестве

GPT-NL: Developing a Dutch LLM from scratch - Julio A. de Oliveira Filho, Athanasios Trantas

GPT-NL will be a language model for Dutch language and culture. It will be transparent, inclusive, and compliant with Dutch and European values, with a platform where we can openly communicate about data and development. GPT-NL is an initiative of TNO, SURF, and NFI. As the training for the GPT-NL model approaches its end, this talk will share the experience and challenges of training a major LLM model from scratch. We will bring insights from our development pipeline, with details on our data curation pipeline and training framework. The session will also explore real-world applications we envision for the model and discuss how the consortium plans to responsibly disseminate GPT-NL within Dutch society.