A blog by Christoph Peters
We propose Gaussian point splatting, a stochastic method to render Gaussian splats that scales extremely well to scenes with many Gaussians. Our core idea is to sample pixel-sized, opaque points from the Gaussians and to splat them to a framebuffer using 64-bit atomics. Through parallel programming primitives, we achieve an even distribution of the workload across millions of threads. Since these…
We are in a transition period, where ray tracing and rasterization coexist. Typical real-time rendering pipelines still use rasterization (often with deferred shading) for primary visibility and then trace rays for shadows, reflections or global illumination. Though, we are getting to the point where one can seriously consider to ditch rasterization and to use ray tracing for primary visibility as…
A core operation in Monte Carlo volume rendering is transmittance estimation: Given a segment along a ray, the goal is to estimate the fraction of light that will pass through this segment without encountering absorption or out-scattering. A naive approach is to estimate optical depth using unbiased ray marching and to then use exp(- ) as transmittance estimate. However, this strategy…
Now that we have a spectral renderer, it is time to see if it was worth the effort. I hope you agree with my reasoning in part 1, that RGB rendering does not match the physical reality, but does it really matter? We will now compare RGB rendering to spectral rendering using my spectral path tracer. In the process, we will look at various types of light sources. RGB rendering simply multiplies RGB…
Based on the insights from part 1, we have the means to define illuminant and reflectance spectra for our scene data. Then the color of a pixel arises from an integral over a product of spectra. The main goal of this blog post is to find efficient ways to evaluate such integrals using Monte Carlo integration and importance sampling. Additionally, we discuss how to define a BRDF based on known…
In my previous blog post, I explained spectral radiometric quantities and a few basics of spectral rendering. In this blog post series, we dive into much more detail on spectral rendering. I present a specific way to implement it in a real-time path tracer and demonstrate what advantages it brings compared to RGB rendering. RGB rendering is still by far the most common way to do rendering. Colors…
Rather than treating colors as RGB triples, spectral rendering uses spectra, which specify an intensity for each wavelength. This blog post series describes how spectral rendering can be implemented at a relatively low overhead (as compared to RGB rendering) and why that is beneficial.
Lightmaps are popular for precomputed global illumination, but require costly recomputation when the scene changes. We present the theory and an iterative algorithm to update lightmaps efficiently, when objects are inserted or removed. Our method is based on path tracing, but focuses on updates to those paths that are affected by the scene change. Using an importance sampling scheme, our solution…
Physically based differentiable rendering has established itself as key to inverse rendering, in which scenes are recovered from images through gradient-based optimization. Taking the derivative of the rendering equation is made difficult by the presence of discontinuities in the integrand at object silhouettes. To obtain correct derivatives w.r.t. changing geometry, accounting e.g. for changing…
This poster presents formulas to compute partial derivatives of any order for the spherical harmonics basis. The key insight is that intermediate results from the usual evaluation of spherical harmonics provide everything that is needed for this purpose. Therefore, the computational cost is negligible. The supplemental includes a code generator for GLSL, HLSL, C, C++ and Python.
We present Splatshop, a highly optimized toolbox for interactive editing (selection, deletion, painting, transformation, ...) of 3D Gaussian Splatting models. Utilizing a comprehensive collection of heuristic approaches, we carefully balance between exact and fast rendering to enable precise editing without sacrificing real-time performance. Our experiments confirm that Splatshop achieves these…
The advantages of spectral rendering are increasingly well known, and corresponding rendering algorithms have matured. In this context, spectral images are used as input (e.g., reflectance and emission textures) and output of a renderer. Their large memory footprint is one of the big remaining issues with spectral rendering. Our method applies a cosine transform in the wavelength domain. We then…
The radiometric quantities introduced in part 1 of this series are completely color-blind. That is an obvious drawback for rendering. Thus far, our notion is that radiance is a single number, not a color. A fairly common practice in rendering is to have separate radiance values for red, green and blue. If your scene happens to be lit exclusively by light sources with only three wavelengths and…
Radiometric quantities are crucial for physically-based rendering. These physical quantities enable us to formulate how much light there is in a scene, where it is and which way it is going. In the end, rendering is all about figuring out what light reaches a camera and how. Without radiometry, there is no way to formalize this problem. For example, the rendering equation, which is the backbone of…
Physically-based rendering requires physical quantities, specifically radiometric and photometric quantities. They are at the foundation of the rendering equation. Without a solid understanding, you cannot properly describe what you are trying to compute in your renderer, let alone compute it correctly and efficiently. In this short blog post series, I describe these concepts in ways that are a…
Earlier this year, I prepared lectures on path tracing for master students at TU Delft. We decided to make recorded versions of these lectures available to the general public. I also wrote a simple Vulkan path tracer for illustrations in the lectures, which is open source now. You can watch the lectures on YouTube (113 minutes) to learn about basic principles of path tracing and all the importance…
A recent paper of mine performs an intersection test in a ray tracing shader. To this end, I had to compute all real roots of a polynomial of moderately high degree (10 to 26). Overall, computing polynomial roots is an extremely well-studied problem where many highly accurate and reliable methods are available [Press2007]. However, implementations of such methods on GPUs are rarely found. After…
Spherical harmonics glyphs are an established way to visualize high angular resolution diffusion imaging data. Starting from a unit sphere, each point on the surface is scaled according to the value of a linear combination of spherical harmonics basis functions. The resulting glyph visualizes an orientation distribution function. We present an efficient method to render these glyphs using ray…
In recent work, Nanite has demonstrated how to rasterize virtualized micro-poly geometry in real time, thus enabling immense geometric complexity. We present a system that employs similar methods for real-time ray tracing of micro-poly geometry. The geometry is preprocessed in almost the same fashion: Nearby triangles are clustered together and clusters get merged and simplified to obtain…
Now that GPUs have ray tracing units, real-time path tracing is coming into reach. Applications beyond movie rendering and baking embrace it, and therefore more people need to know about it. At Intel, I recently offered a path tracing workshop to educate a broad audience of engineers on basics of the topic. I am happy to announce that we decided to make this workshop publicly available. If you…
Recent advances in ray tracing hardware bring real-time path tracing into reach, and ray traced soft shadows, glossy reflections, and diffuse global illumination are now common features in games. Nonetheless, ray budgets are still limited. This results in undersampling, which manifests as aliasing and noise. Prior work addresses these issues separately. While temporal supersampling methods based…
I was not planning to have a fifth part in this series about my toy renderer but there were some interesting changes for our paper on vertex-blend attribute compression. The renderer now supports animations with linear vertex-blend animation (also known as skinning). That means there is a skeleton consisting of animated bones and those influence the mesh. I had to export these animations from…
Compression of vertex attributes is crucial to keep bandwidth requirements in real-time rendering low. We present a method that encodes any given number of blend attributes for skinning at a fixed bit rate while keeping the worst-case error small. Our method exploits that the blend weights are sorted. With this knowledge, no information is lost when the weights get shuffled. Our permutation coding…
We present a novel image-based representation to interactively visualize large and arbitrarily structured volumetric data. This image-based representation is created from a fixed view and models the scalar densities along each viewing ray. Then, any transfer function can be applied and changed interactively to visualize the data. In detail, we transform the density in each pixel to the Fourier…
When people look at my shader code, one of the most frequently asked questions is why I use the GLSL fma instruction (or its HLSL equivalent mad) so frequently. In spite of the punny title of this post, fma actually stands for fused multiply-add, i.e. it implements the formula fma(a,b,c)=a*b+c. It is faster than separate multiplication and addition because on most CPUs and GPUs it counts as one…
I used to make my slides in Powerpoint. For my presentations at SIGGRAPH 2021 and HPG 2021, I tried something different. The whole slide deck is one big interactive matplotlib plot. Every bit is scripted in Python. They feature lots of animated and interactive plots. I am satisfied with how this experiment worked out. Lazily hacking together slides works better with Powerpoint but if you want a…
In the past winter term at Karlsruhe Institute of Technology, I had the pleasure of supervising Bastian Urbach's bachelor thesis. His topic has been the generalization of methods for shading with polygonal lights to polyhedral lights. He has been highly motivated and creative. The result is an efficient method for GPU-accelerated real-time shading with convex or non-convex polyhedral lights (see…
Part 4 of this series about my toy renderer is all about ray traced shadows. In particular, I discuss direct lighting with polygonal and linear lights for diffuse and specular surfaces. I have described the corresponding methods in two recent research papers but this post is written with a broader audience in mind. While the papers focus on the mathematical derivation of the algorithms, this post…
Part 3 of this series about my toy renderer covers some basic techniques for rendering. My renderer does nothing fundamentally new on this front but some choices are a bit exotic. It is ultimately about real-time ray tracing, so everything should play nicely with that. Besides I want it to be slick and fast. Disregarding dear ImGui, the whole thing only makes two draw calls per frame. It uses a…
Part 2 of this series about my toy renderer is all about scene data. The requirements are fairly lax, since I only care about static triangle meshes. For the most part, I want to be able to render the ORCA assets and a couple of scenes from Blendswap. However, I emphasized how much long compile times harm productivity in the previous post. All of that applies equally to load times since they…
With the advent of real-time ray tracing, there is an increasing interest in GPU-friendly importance sampling techniques. We present such methods to sample convex polygonal lights approximately proportional to diffuse and specular BRDFs times the cosine term. For diffuse surfaces, we sample the polygons proportional to projected solid angle. Our algorithm partitions the polygon suitably and…
We introduce an efficient method to sample linear lights, i.e. infinitesimally thin cylinders, proportional to projected solid angle. Our method uses inverse function sampling with a specialized iterative procedure that converges to high accuracy in only two iterations. It also allows us to sample proportional to a linearly transformed cosine. By combining both sampling techniques through suitable…
Part 1 of this series about my toy renderer covers the most fundamental design decisions. Over the years, I have written many renderers and for a long time their complexity kept growing. This time, I took the opposite route. I wanted to maximize the fraction of code that implements crucial functionality rather than wasting my time on bloaty infrastructure. The code that I wrote (excluding shaders)…
Alongside my latest papers I released the underlying renderer as open source. It is a real-time deferred renderer with ray traced shadows based on Vulkan and written in C. As I wrote it, I had the liberty to try some unconventional designs and techniques. So I did, because that is an excellent way to learn new things. It also became the basis for our work on vertex-blend attribute compression.
In my job at the Karlsruhe Institute of Technology, I usually supervise two bachelor or master theses per term (four per year). By handing out topics that have a sufficiently narrow scope but tap directly into current rendering research, I try to pass on my passion for this subject.
Rapidly growing data sizes of scientific simulations pose significant challenges for interactive visualization and analysis techniques. In this work, we propose a compact probabilistic representation to interactively visualize large scattered datasets. In contrast to previous approaches that represent blocks of volumetric data using probability distributions, we model clusters of arbitrarily…
If you are familiar with my work, you may be surprised to hear that I barely ever write documents in LaTeX. All of my first author papers, as well as my bachelor, master and PhD thesis are written entirely in LyX. LyX is a text editor that generates LaTeX code for you. I appreciate it because it helps me get through the creative process of writing without distraction and aids my mathematical…
Over the past year, I have gotten dissatisfied with the setup of my blog. Discoverability of posts was not particularly good, maintenance work began to outgrow the work for new content and I grew tired of the Wordpress layout. Rather than fixing each issue individually, I decided to turn the whole thing into a static HTML page generated by my own set of Python scripts. The resulting redesign will…
We propose a data reduction technique for scattered data based on statistical sampling. Our void-and-cluster sampling technique finds a representative subset that is optimally distributed in the spatial domain with respect to the blue noise property. In addition, it can adapt to a given density function, which we use to sample regions of high complexity in the multivariate value domain more…
We present a compact and efficient representation of spectra for accurate rendering using more than three dimensions. While tristimulus color spaces are sufficient for color display, a spectral renderer has to simulate light transport per wavelength. Consequently, emission spectra and surface albedos need to be known at each wavelength. It is practical to store dense samples for emission spectra…
In a recent journal paper, we introduced a technique to represent reflectance spectra by an arbitrary number of Fourier coefficients. As a special case, we converted tristimulus data to three Fourier coefficients. After summarizing this work, we introduce the Fourier sRGB color space. It is defined in terms of Fourier coefficients but designed to behave similar to sRGB. Textures stored in Fourier…
This blog post answers a question that Tom Davidovi from Weta Digital had about the recently published projected solid angle sampling for spherical caps. In Monte Carlo rendering, it is very common to combine several sampling techniques through multiple importance sampling. In this case, any of the sampling techniques may produce a sample and then you need to compute the probability density for…
Stochastic shading with area lights requires methods to sample the light sources. For diffuse materials, the best strategy is to sample proportionally to projected solid angle. Recent work in offline rendering has addressed this problem for spherical light sources, but the solution is unsuitable for a GPU implementation. We present a far more efficient solution. It offers results without…
Back in May, we published moment-based order-independent transparency (MBOIT) at the Symposium on Interactive 3D Graphics and Games 2018. This brief post follows up on two things; a similar but independent work and some missing code.
With the push towards physically based rendering, stochastic sampling of shading, e.g. using path tracing, is becoming increasingly important in real-time rendering. To achieve high performance, only low sample counts are viable, which necessitates the use of sophisticated reconstruction filters. Recent research on such filters has shown dramatic improvements in both quality and performance. They…
Compositing transparent surfaces rendered in an arbitrary order requires techniques for order-independent transparency. Each surface color needs to be multiplied by the appropriate transmittance to the eye to incorporate occlusion. Building upon moment shadow mapping, we present a moment-based method for compact storage and fast reconstruction of this depth-dependent function per pixel. We work…
Wir wenden die Theorie der Momente auf Probleme des Visual Computings an. Aus dieser Theorie entwickeln wir effiziente Algorithmen, die eindimensionale Verteilungen durch eine geschlossene Form aus ihren Momenten rekonstruieren. Solche Rekonstruktionen nutzen aus, dass die urspr nglichen Verteilungen keine negativen Massen beinhalten. Dadurch k nnen sie vor allem bei Verteilungen, die um wenige…
My dissertation has been published digitially and is now available in its entirety as a free download. Most results have been published before through my papers at i3D 2015, SIGGRAPH Asia 2015 and JCGT. Though, there is some entirely new material. Since I spent a lot of time writing it, there better be somebody to read parts of it. Below I'll try to wet your appetite.
We apply the theory of moments to develop computationally efficient methods for real-time rendering of shadows and reconstruction of transient images from few measurements. Given moments of an unknown probability distribution, i.e. the expectations of known, real random variables, the theory of moments strives to characterize all distributions that could have led to these moments. Earlier works in…
My recent paper on non-linearly quantized moment shadow maps promises an executable demo. Preparing that took a little longer than expected but to make up for the delay, the demo has plenty of new features. Most notably it now uses dear imgui and includes the applications of blue noise I blogged about earlier. Rather than showing off the new technique only, this demo is an extension of earlier…