Recap on fine-tuning to follow instructions
Workbook notes on instruction fine-tuning a pretrained GPT-2 model in Elixir.
Karlo Šmid Blog
karlosmid.github.io
Latest posts
Fine-tuning to follow instructions
Fine-tuning a pretrained GPT-2 model in Elixir so it can follow instructions, generate task-specific responses, and be evaluated with Ollama.
Recap of fine-tuning for classification
Workbook notes on fine-tuning a pretrained GPT-2 model for SMS spam classification in Elixir.
Fine-tuning for classification
This post fine-tunes a pretrained GPT-2 model for SMS spam classification in Elixir.
Recap of training a GPT model
Workbook notes on GPT training, cross-entropy loss, perplexity, and model weights.
Pretraining on unlabeled data in Elixir
A practical Elixir walkthrough of GPT pretraining: calculating generation loss, training on The Verdict, controlling decoding with temperature and top-k sampling, saving checkpoints, and loading OpenAI GPT-2 weights.
Estimating the population of Zabok using public data
The last public population census in Croatia was done in 2021. With the latest development in Zabok, I made a simple estimate of the current population of Zabok.
Recap on implementing GPT model from scratch
TL;DR I am recaping blog post Implementing a GPT model from scratch by answering workbook questions Build llm from scratch chapter 4, and reading Giles blog posts on this chapter: part15 part16 part17 part18 part19 Giles’ blog posts I got lost during Giles’ part 15 on the first read. The second read made things clearer. He goes deep into the GPTModel architecture and how data flows from tokens to…
Implementing a GPT model from scratch
This post implements a GPT-style model from scratch in Elixir/Nx.
AWS error: not authorized to perform
A misleading Amazon Connect authorization error turned out to be an API request formatting issue. Comparing the call with AWS CLI output and checking CloudTrail revealed that StartTime and EndTime needed epoch timestamp values.