Personal site of Sayak Paul.
A few days back, AMD gifted me a GMKtec EVO-X2 so that I could do some of my compute-heavy work locally. NVIDIA did something similar with their DGX Spark a few months back, and using it in my regular tasks has been very productive. For those who’re unfamiliar with EVO-X2, it’s a mini-PC equipped with an AMD Ryzen AI Max+ 395 (also known as “Strix Halo”) processor, ideal for local AI development…
This post has an accompanying talk I gave at the MIT BCS department. Watch the video here . For the past few years, the space of synthetic image and video generation has been on the rise. The results have been nothing short of extraordinary, and they continue to get better. At the center of this revolution lies a class of models – flow-matching – known for its unique framework to connect noise to…
Attention is a crucial component in generative neural architectures for continuous modalities like images and videos from natural language. More specifically, cross-attention helps to contextualize the relationship between the natural language prompt inputs and the media, being generated. With modern diffusion models (or shall we say “flow”) for condition-guided image and video generation, we saw…
Releasing a new version of an open-source library is an exhilarating experience. You ship new features, bug fixes, improved documentation, etc., to serve your users and also the mission of your library. Being one of the maintainers of the 🧨 Diffusers library , I am no exception to this. Once a release is finalized, it’s usually published on a software repository for the distribution programming…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . In this article, highlight the pain points of provisioning hardware manually (especially for ML experiments), discuss how they originate, and how we can address these issues by provisioning hardware with code. We’ll cover this using Vertex AI Workbench…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . In ecommerce, accurate product categorization is important in order to provide contextual search results, effective filtering, relevant reporting, and product recommendations. At Carted, we’re building tools to enable developers to build seamless shopping…
Many people who aspire to become Machine Learning (ML) practitioners find it particularly difficult to continue to hone relevant skills when they pursue a job that does not involve even a tiny bit of ML. So, if you’re serious about choosing ML as a potential career option, it’s important to ensure you continue to practise what you’re learning along the way. Otherwise, there’d likely be nothing for…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . This article is part of a three-part series on handling variable-length text sequences in TensorFlow: Part 1: Optimizing Sequence Padding Part 2: Training a Simple BERT Model Part 3: Using a Sentence-Conditioned BERT Encoder (you are here) In Part 2 of…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . This article is part of a three-part series on handling variable-length text sequences in TensorFlow: Part 1: Optimizing Sequence Padding Part 2: Training a Simple BERT Model (you are here) Part 3: Using a Sentence-Conditioned BERT Encoder In Part 1 of…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . This article is part of a three-part series on handling variable-length text sequences in TensorFlow: Part 1: Optimizing Sequence Padding (you are here) Part 2: Training a Simple BERT Model Part 3: Using a Sentence-Conditioned BERT Encoder Text data comes…
In this post, I discuss my perspective on two primary questions pertaining to the Google Summer of Code (GSoC) program . Even though my work is centered around Machine Learning (ML), I believe these pointers are domain-agnostic. This is based on my experience of mentoring for TensorFlow at GSoC 2021 . Please note that these thoughts are of my own and may not reflect what anyone else (including the…
This work was done at Carted , where I worked as a (founding) machine learning engineer. The post was originally published on the Carted blog . Data scale is crucial at Carted. The machine learning (ML) capabilities we’re building need internet-scale data to power them. With such large-scale data regimes comes an obvious challenge of careful engineering that allows us to handle these regimes…
I recently added 15 different variants of the ConvNeXt architecture to TensorFlow Hub (TF-Hub). This post is a reflection of what had to be done to get to that point. First, we’ll discuss the implementation of ConvNeXt in Keras and how the original pre-trained parameters were ported into these models. We’ll then talk about TF-Hub’s ConvNeXt collection and what it offers. I hope this post is useful…
This post shows how to build and install OpenCV 4.5.0 on a MacBook Pro that comes with an M1 chip . Yes, you guessed it right - as of January 01, 2021 , there’s no pre-compiled OpenCV binary compatible with this MacBook Pro variant. So, open up a terminal and get started! Here’s a brief summary of the configuration of my MacBook - Note The following steps should run well on an M1 MacBook Air too.…
In the previous post , we saw how to convert the pre-trained CRAFT model from PyTorch to TensorFlow Lite (TFLite) and run inference with the converted TFLite model. In this post, we will be comparing the TFLite variants of the CRAFT model to another text detection model - EAST . The objective of this post is to provide a comparative study between these two models with respect to various…
This year researchers from the University of Wisconsin-Madison and Google published their work on MobileDet . MobileDet presents an architectural philosophy for designing object detectors specifically targeted toward running on mobile accelerators like DSP, EdgeTPU, and so on. MobileDet yields significant improvement over architectures MobileNetV2+SSDLite and MobileNetV3+SSDLite on the COCO object…
Data augmentation is a favorite recipe among deep learning practitioners especially for the ones working in the field of computer vision. Data augmentation is a technique used for introducing variety in training data thereby helping to mitigate overfitting. When using Keras for training image classification models, using the ImageDataGenerator class for handling data augmentation is pretty much a…