I'm an independent AI engineer helping companies build robust AI-powered products.
Giving an LLM the capability to call some external function based on the user’s input and receive the results back is a very powerful pattern and a key element behind the rapid rise of agentic workflows. This pattern powers many of the features we see on ChatGPT today, such as web search, code execution, image generation, or personalized memory based on conversation history. LLM providers expose…
Synthetic data is a popular approach for bootstrapping an initial dataset when building LLM-based applications. We can find practical examples of synthetic data usage in the wild such as: Generating synthetic user queries from existing documents to evaluate RAG systems 1 Producing fake meeting transcripts for video call summarization 2 Bootstrapping lots of texts (emails, inquiries, multi-turn…
Background I recently built a custom feed on Bluesky to capture the latest discussions on pre-prints from arxiv.org and research papers from conferences like ACL. It was inspired by this bluesky post from a researcher requesting for such a feed. While there are drag-and-drop custom feed generators like Skyfeed , you are limited to using only regular expressions for the filtering part. If you use a…
tqdm is a popular library that’s widely used in a bunch of open-source python ML libraries for displaying progress bars. As such, it’s already pre-installed as a dependency when working on machine learning projects. shell pip show tqdm Required-by: datasets , dvc , evaluate , huggingface-hub , openai , sentence-transformers , spacy , transformers For example, consider a task where we loop over a…
It’s a common task in NLP to either check a text against a pattern or extract parts from the text that matches a certain pattern. A regular expression or “regex” is a powerful tool to achieve this. While powerful, regex can feel daunting as it comes with a lot of features and sub-parts that you need to remember. In this post, I will illustrate the various concepts underlying regex. The goal is to…
Self Supervised Learning is an interesting research area where the goal is to learn rich representations from unlabeled data without any human annotation. This can be achieved by creatively formulating a problem such that you use parts of the data itself as labels and try to predict that. Such formulations are called pretext tasks. For example, you can setup a pretext task to predict the color…
Embedding Projector is a free web application for visualizing high-dimensional data. It has built-in demos for visualizing word embeddings in NLP and image embeddings for MNIST in Computer Vision. I recently experimented with a way to load sentence embeddings along with the class labels into this tool and explore them interactively. In this blog post, I will explain the end-to-end process with an…
I recently discovered a way to set up VSCode on Google Colab and use it as an editor to write code and run experiments on the Colab VM. With this setup, you can still prototype in the Colab Notebook while also using VSCode for all the advantages of a full-fledged code editor. Here is how you can replicate my setup. Approach 1: Python Package In this setup, we use the colab-code package that…
Hugging Face recently released 1008 translation models for almost 140 languages on their model hub. These models were originally trained by Jörg Tiedemann of the Language Technology Research Group at the University of Helsinki . They were trained on the Open Parallel Corpus(OPUS) using a neural machine translation framework called MarianNMT . In this post, I will explain how you can use the…
Keyword Extraction is one of the simplest ways to leverage text mining for providing business value. It can automatically identify the most representative terms in the document. Such extracted keywords can be used for various applications. They can be used to summarize the underlying theme of a large document with just a few terms. They are also valuable as metadata for indexing and tagging the…
Most software products we encounter today have some form of search functionality integrated into them. We search for content on Google, videos on YouTube, products on Amazon, messages on Slack, emails on Gmail, people on Facebook, and so on. As users, the workflow is pretty simple. We can search for items by writing our queries in a search box and the ranking model in their system gives us back…
When developing an NLP model, it’s a standard practice to test how well a model generalizes to unseen examples by evaluating it on a held-out dataset. Suppose we reach our target performance metric of 95% on a held-out dataset and thus deploy the model to production based on this single metric. But, when real users start using it, the story could be completely different than what our 95%…
Semi-supervised learning methods for Computer Vision have been advancing quickly in the past few years. Current state-of-the-art methods are simplifying prior work in terms of architecture and loss function or introducing hybrid methods by blending different formulations. In this post, I will illustrate the key ideas of these recent methods for semi-supervised learning through diagrams. 1.…
While Flask has become the de-facto choice for API development in Machine Learning projects, there is a new framework called FastAPI that has been getting a lot of community traction. I recently decided to give FastAPI a spin by porting a production Flask project. It was very easy to pick up FastAPI coming from Flask and I was able to get things up and running in just a few hours. The added…
Colab is one of the best products to come from Google. It has made GPUs freely accessible to learners and practitioners like me who otherwise wouldn’t be able to afford a high-end GPU. While the interface is very easy to use, there are many lesser-known and undocumented features in colab. In this post, I will share those features that I’ve discovered from basic usage and their official talks. 1.…
Word Embeddings are one of the most interesting aspects of the Natural Language Processing field. When I first came across them, it was intriguing to see a simple recipe of unsupervised training on a bunch of text yield representations that show signs of syntactic and semantic understanding. In this post, we will explore a word embedding algorithm called “FastText” that was introduced by…
With transformer models such as BERT and friends taking the NLP research community by storm, it might be tempting to just throw the latest and greatest model at a problem and declare it done. However, in industry, we have compute and memory limitations to consider and might not even have a dedicated GPU for inference. Thus, it’s useful to keep simple and efficient models in your NLP…
I recently completed the UC Berkeley’s Deep Unsupervised Learning course. The course had an interesting guest lecture on the history of language modeling by Alec Radford, the author of GPT model. In one of his slides, Alec mentions how by simply observing a bunch of strings, language models tend to capture useful knowledge. He also mentions that maybe in the future, we could have an unsupervised…
In my last post , we explored a contrastive learning approach to zero-shot text classification. In this post, we will explore a different approach based on text generation. This approach was proposed by Puri et al. in their paper “Zero-shot Text Classification With Generative Language Models” . The paper was also presented in the “3rd Workshop on Meta-Learning” at NeurIPS 2019. The goal of…
The recent release of GPT-3 got me interested in the state of zero-shot learning and few-shot learning in NLP. While most of the zero-shot learning research is concentrated in Computer Vision, there has been some interesting work in the NLP domain as well. I will be writing a series of blog posts to cover existing research on zero-shot learning in NLP. In this first post, I will explain the paper…