Beyond Tokenization: The Four Taxes and the Path Forward
The compounding tax stack low-resource languages carry, why vision encoders might hold the key, and the open research questions.
Omar Kamali
I write about technical concepts in human terms.
Latest posts
The Hidden Tax Your LLM Pays for Bad Tokenization
How bad tokenization forces language models to waste capacity on reconstruction instead of reasoning.
Tokenization is Killing Our Multilingual LLM Dream
Why tokenization is the hidden bottleneck blocking truly multilingual AI — lessons from building Sawalni and Wikilangs.
Why I stopped trusting the official Wikipedia dataset, and what I did about it
It all started with a DM from a friend, member and contributor to the Moroccan Wikipedia community. "Are you using the current version of Wikipedia? The official dataset is severely outdated. We added so many cool articles nowhere on huggingface" He was right. I was running a 2023 snapshot in 2025.
A Wordle for the Worldle
I built a word game for more than 300 languages, each drawing on its own Wikipedia as the source. Here's the thing nobody tells you: building a simple word game for most of these languages meant building things that didn't exist.
Picomon 0.2.0: From AMD Crash Fix to GPU Monitoring That Doesn’t Suck
Earlier this month, I whipped up a Python script with an LLM that parsed amd-smi output. It was ugly. It worked. I called it picomon.
Introducing Wikipedia Monthly: Fresh, Clean Wikipedia Dumps for NLP & AI Research
Announcing Wikipedia Monthly, an always fresh dataset to support research for low-resource languages
Getting Perfectly Structured Data from LLMs
If you've ever struggled to get consistent JSON output from large language models, I have a simple and clever solution for you.
2024: A Year of Growth, Innovation, and Community
As we leave 2024 behind, I found myself reflecting over the holidays on a transformative year that reshaped my grasp of technology's role in human connection.
Datapluck: Portability Tool for Huggingface Datasets
Exporting & importing Hugging Face datasets to spreadsheets and various file formats.
OpenAI RAG: A Step Forward or a Step Too Far?
OpenAI's data integrations in Assistant and GPTs is causing ripples in the AI world. Beyond the excitement, let's look at OpenAI's strategy critically, the tension in its ecosystem, questionable comparisons to Apple, and the impending threat of commodification that OpenAI itself may face.
Web Scraping: Data For Everyone
We know what Data is, but where does it come from? With Web Scraping you can collect data from any website. This article will get you started in this world.
What is Data: A Brief Guide
An introduction to ease you into the world of data, what it is, what is it useful for and privacy concerns, as a preamble to the Data Series.
Shaping the Future with Sawalni: The Dawn of Moroccan AI
I've been asked multiple times, "Why are you creating a Moroccan AI?" Today I want to share the story behind Sawalni, the first AI in history to speak our beautiful Moroccan Darija, with all of you.
Tokens, the secret language of AI
Have you ever wondered how AI systems make sense of the vast amount of information they encounter? Let's look at AI tokens and why you should care.
DNS for non-Techies: Domains, Website Performance, and Content restrictions
New websites are launched every day and others stop existing overnight. Servers and IP addresses get repurposed. How does DNS keep it all together?
How I built my newsletter using Gmail and Google Sheets
I've recently wanted to add a newsletter to my blog. I decided to build it instead of using a marketing tool.
It starts with a message: Internet, DNS & Networks for non-Techies
There are over 22 billion machines connected to the internet. How do these machines communicate with each other to fulfil their purpose?
What happens when you open a TikTok?
Do you ever stop and think about the countless invisible components working behind the scenes to bring an app like TikTok to life on your phone?