Personal blog of Khushi Agrawal
Introduction A reduction is a method of deriving a single value from an array of values. For example, the summation of an array. A parallel reduction is a technique of coordinating parallel threads to produce the right results. The reduction can be defined for mathematical operations like addition, subtraction, min, max, multiplication, etc. This blog post will start with reduction trees, a simple…
Introduction The aim of the blog posts is to introduce a parallel histogram pattern, where each output element can be updated by any thread. Therefore, we should coordinate among threads as they update the output value. In this blog post, we will read the introduction about using atomic operations to serialize the updates of each element. Then, we will study an optimization technique:…
Introduction Hey there! In this post, we’ll start by introducing the stencil operation, covering its background, basic algorithm, and a series of optimization techniques such as shared memory tiling for stencil sweeps, thread coarsening, and register tiling. Stencils are used to solve partial differential equations (PDEs) in applications such as fluid dynamics, heat conductors, weather…
Introduction Convolution is an array operation used in various forms in signal processing, digital recording, image/video processing, and computer vision. In convolution, each output element is calculated independently as a weighted sum of the corresponding and surrounding input elements. These weights used in calculating the weighted sum are known as a filter array and are defined as a…
Introduction The aim of the blog posts is to explain how to achieve high-performance computing. We need to manage parallel code alongside the given hardware resources. We’ll read about the off-chip memory architecture and discuss memory coalescing, memory latency hiding, and thread coalescing (which depends on the different aspects of the architecture). Lastly, we’ll study the common checklist of…
Introduction Hi there! The blog posts aim to study the GPUs’ on-chip memory architecture and how to organize and position data for efficient thread access. Until now, we executed our programs all from global memory access (off-chip DRAM), which leads to delays and traffic and negatively affects performance. In this blog post, we will study ways to tolerate these long-latency operations, i.e. we’ll…
Introduction Hi there! The blog post aims to share a high-level overview of the computing architecture. It then explores the concepts of resource alignment, block scheduling, and occupancy. It also discusses thread scheduling, latency tolerance, control divergence, and synchronization. This blog post is written while reading the fourth chapter, Compute Architecture and Scheduling, of the…
Introduction Hi there! The blog posts aim to share how the blocks and threads are organized and how they are used to process multidimensional data. We’ll start with the basics of thread organization and then move our focus to its applications. We’ll demonstrate color-to-gray scale conversion, blurring images and a naive implementation of matrix multiplication in CUDA. This blog post is written…
Hi there! I’ve been diving into the world of parallel programming from the excellent book “Programming Massively Parallel Processors: A Hands-on Approach 1 ” by Wen-mei W. Hwu , David B. Kirk , and Izzat El Hajj . The content of this blog post is motivated by the book’s first chapter: Introduction, the second chapter: Heterogeneous data-parallel Computing, and the Intel blogs 2 on vectorization. I…
Hi, Guys! In this post, we will go through the concepts of Lambda expression in C++. That’s a pretty interesting thing. Before starting my blog, I want to present my gratitude to Arthur O’Dwyer for his fantastic talk on Back to Basics: Lambdas from Scratch - CppCon 2019 . Thanks so much! The content of the blog posts is entirely inspired by his talk. Let’s start with the fundamental implementation…