Ginjinha: Pretraining LLMs on European Portuguese

For the past year, I've been very curious about the intersection of Large Language Models (LLMs) and the European Portuguese language. I've built evaluations , classifiers , and eventually the largest pretraining dataset in European Portuguese . Now, that's all fine and dandy, but it raises the obvious question: Can we pretrain an …