Hey, I am Jackson! I’m an AI engineer specializ...
TL;DR: By combining Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) , we achieved ~73% accuracy on the Qwen 2.5 0.5B Instruct language model—even though it was never trained for vision tasks before! A month ago, DeepSeek R1 was open-sourced, highlighting how pure reinforcement learning (RL) can enhance LLM reasoning through test-time compute. A key part of this is Group…
It's been some time since my last post here. The world of AI doesn't seem to slow down, and it's always full of new things. It's amazing how quickly things change; sometimes, missing a day or two of updates on Twitter (should be X, my primary source of research news) can make me feel a bit out of the loop. The challenge these days is not just keeping up, but choosing wisely what to delve into…
Introduction Previously, we discussed Low-Rank Adapters ( LoRA ) as a method for efficiently fine-tuning large language models (LLMs). In this post, we will discuss QLoRA , a new quantization method that builds on LoRA to enable even more efficient LLM fine-tuning. QLoRA reduces the memory requirements for fine-tuning a 65 billion parameter LLM from over 780GB to less than 48GB of GPU memory. This…
Introduction Recently, many efforts have emerged aiming to expand the context length of Large Language Models (LLMs), all while keeping their perplexity stable. Among these initiatives, a noteworthy paper titled LongLoRA stands out. It impressively manages to extend the context size of the Lama-70b model to 32k and the more compact Llama-7b model to a whopping 100k! And guess what? It accomplishes…
Given the rapid advancements in large language models (LLMs) like the recent launch of Llama 2 and research focusing on parameter efficiency, hallucination reduction, and accelerated inference, we're seeing the gap close between open-source models and commercial solutions like ChatGPT. Companies today often don't require a Jack-of-all-trades language model capable of solving complex mathematical…
Introduction With the launch of cutting-edge models like ChatGPT, the world has been witnessing a remarkable shift towards the development of Large Language Models (LLMs). Take, for example, Meta's recent release of LLama 2, an open-source LLM with a staggering 70 billion parameters. Now, even if you were to harness the most powerful GPU available today, the A100 with an 80GB memory capacity, this…
Transformers Ever tried naming a Large Language Model (LLM) that doesn't have a Transformer hiding under its hood? I'd bet you'd have an easier time getting a cat to walk on a leash without transforming into a furry tornado of chaos. The ubiquity of Transformers in this field isn't mere coincidence, but they have reshaped the landscape of natural language processing due to their ability to…
What is self-supervised learning? Self-supervised learning, a learning paradigm passionately advocated by Meta AI's VP and Chief AI Scientist, Yann LeCun, provides a remarkable avenue for models to acquire meaningful representations directly from unlabeled data. This paradigm offers a promising solution to the high cost and labor-intensive nature of annotated data, as models can leverage the…
What is text-to-image model personalization? In a world where memories are precious and photos hold cherished moments, there are times when we yearn for more. Imagine having the power to bring your loved ones back to life, not just through existing photographs, but by recreating them in entirely new scenes. It's now possible with the exponential grow of AI. With just a handful of example images,…