RSSAmplifier

Blog

Alessio Devoto

Alessio Devoto's PhD Data Science personal website

alessiodevoto.github.io ↗RSS feed ↗10 posts

Latest posts

(untitled)

In this short tutorial, we will use our company’s dataset to predict whether a customer will churn or not. For a better experience, open in Colab:

Your First Object-Oriented Agent

Build a BaristaAgent with NOOA — tools, typed outputs, and state in plain Python.

Peeking Inside Diffusion Language Models with LogitLens

Lately, diffusion-based language models like LLaDA and MMaDA have been gaining traction. These aren’t your standard left-to-right text generators - they’re bidirectional models trained to fill in missing tokens, more akin to BERT but on steroids. During training, Diffusion Language Models (DLMs) learn to predict <mask> tokens given context, effectively learning a denoising task.

Visualizing the Vocabulary of an LLM

Disclaimer: unlike other posts in this blog that actually served some purpose, this is just a random idea I had and am implementing for fun. So if your question is “why should I want to visualize the vocabulary of an LLM?”, I don’t have an answer 😄

LogitLens From Scratch With Hugging Face Transformers

In this short tutorial, we’ll implement LogitLens to inspect the inner representations of a pre-trained Phi-1.5. LogitLens is a straightforward yet effective interpretability method.

Vision Transformer in *pure* JAX.

I decided to do this for two reasons. The first reason is that, for years, I had to bear my Ph.D. advisor coming into the lab while I was happily coding my Pytorch model, slowly sneaking at my back, stare at my screen and say - with a disappointed look - “you should definitely do this in JAX”. The second reason is this nice blog post from Neel Gupta.

Visualizing Attention Maps in Pre-trained Vision Transformers (Pytorch)

Goal: Visualizing the attention maps for the CLS token in a pretrained Vision Transformer from the timm library.

Short Notes on Types of Parallelism for Training Neural Networks

As neural networks grow larger (see LLMs, though now it looks like we also have a trend towards smaller models with Gemma2-2b ) and datasets become more massive, parallelism techniques are crucial for efficient training. This is a short, far-from-exahustive list of different types of parallelism that can be found out there in the wild.

Efficiency Metrics in Machine Learning

In the world of machine learning, efficiency is a buzzword we hear all the time. New methods or models often come with the claim of being more efficient than their predecessors. But what does “more efficient” actually mean? Comparing efficiency objectively can be tricky since the metrics used to measure it are often confusing and varied. Some are hardware-dependent, while others are not. Some…

Flops with Pytorch built-in flops counter

It is becoming more and more common to use FLOPs (floating point operations) to measure the computational cost of deep learning models. For Pytorch users, unfortunately, it looks like there is no agreed upon method or library to do that.