Recent content on Jeff Klukas
Summary of a talk delivered at Apache Beam Digital Summit on August 4, 2021. This session will start with a brief overview of the problem of duplicate records and the different options available for handling them. We’ll then explore two concrete approaches to deduplication within a Beam streaming pipeline implemented in Mozilla’s open source codebase for ingesting telemetry data from Firefox…
Summary of a talk delivered at Apache Beam Digital Summit on August 24, 2020. In this session, you won’t learn about joins or windows or timers or any other advanced features of Beam. Instead, we will focus on the real-world complexity that comes from simply moving data from one system to another safely. How do we model data as it passes from one transform to another? How do we handle…
Originally published as a cookbook on docs.telemetry.mozilla.org to instruct data users within Mozilla how to take advantage of the usage history stored in our BigQuery tables. Monthly active users (MAU) is a windowed metric that requires joining data per client across 28 days. Calculating this from individual pings or daily aggregations can be computationally expensive, which motivated creation…
Cross-posted on the Data@Mozilla blog . The data teams at Mozilla have put a great deal of effort into building a robust data ingestion pipeline and reliable data warehouse that can serve a wide variety of needs. Yet, we keep coming back to conversations about the dashboard problem or about how we’re missing last mile tooling that makes data accessible for use in data products that we can release…
Originally published as an article on docs.telemetry.mozilla.org. So you want to build a data lake… Where do you start? What building blocks are available? How can you integrate your data with the rest of the organization? This document is intended for a few different audiences. Data consumers within Mozilla will gain a better understanding of the data they interact with by learning how the…
Originally published as a guest post on the CircleCI Blog. On public code repositories, arbitrary users are generally allowed to make forks and issue pull requests (which we will refer to as “forked PRs”). These users could be from outside your organization and are generally considered untrusted for the purposes of running automated builds. This causes a problem if you want a Continuous…
Originally published as a guest post on the CircleCI Blog. Continuous integration (CI) tools have been evolving towards flexible, general-purpose computing environments. They aren’t just used for running tests and reporting results, but often run full builds and send artifacts to external systems. If you’re already relying on a CI system for these other needs, it can be convenient to…
Originally published as a guest post on the CircleCI Blog. Mozilla likes to work in the open as much as possible, which means we primarily do our development in publicly accessible code repositories, whether we expect outside collaborators or not. Those repositories, however, still need to hook into other systems, which sometimes involves managing sensitive credentials. How can we enable those…
I’ve recently started working quite a bit with Spark and have found that there’s not much guidance on best practices for packaging and deploying libraries and apps on Spark. I’m planning to write a series of posts on Spark packaging and app deployment as we find patterns that work for the data platform at Mozilla. Spark is written in Scala, but provides client libraries for…
Originally posted on the Simple engineering blog; also presented at PGConf US 2017 and Ohio LinuxFest 2017 We previously wrote about a pipeline for replicating data from multiple siloed PostgreSQL databases to a data warehouse in Building Analytics at Simple , but we knew that pipeline was only the first step. This post details a rebuilt pipeline that captures a complete history of data-changing…
Originally posted on the Simple engineering blog; also presented at PyOhio 2016 and CloudDevelop 2016 Early in 2014, Simple was a mid-stage startup with only a single analytics-focused employee. When we wanted to answer a question about customer behavior or business performance, we would have to query production databases. Everybody in the company wanted to make informed decisions, from…
Originally posted on the Simple engineering blog Simple has adopted Amazon Redshift as a data warehouse. It’s used not just by our data analysts, but by teams throughout the company to answer questions about fraud, customer behavior, and more.
My Ph.D. dissertation, performed at the University of Wisconsin-Madison. Abstract: A search for exotic particles decaying via WZ to final states with electrons and muons is performed using a data sample of pp collisions collected at 7 TeV center-of-mass energy by the CMS experiment at the LHC, corresponding to an integrated luminosity of 4.98 inverse femtobarns. A cross section measurement for the…
The LHC is currently running at 7 TeV, giving a relativistic gamma factor of 3730: $$E = \gamma mc^2 \rightarrow \gamma = {E \over mc^2} \rightarrow \gamma(3.5\text{TeV}) = 3730$$ This means the protons are moving at 99.999996% the speed of light: $$\gamma = {1 \over \sqrt{1 - \beta^2}} \rightarrow \beta = \sqrt{1 - {1 \over \gamma^2}} \approx 0.99999996$$ Alright, now let’s look at the…
This research was performed under an NSF-funded Research Experience for Undergraduates program at Indiana University and then extended at Wittenberg University to serve as an undergraudate honors thesis. The work was overseen by John Beggs. It was later published in the 2006 issue of Wittenberg University’s non-fiction literary magazine, Spectrum .
This research was performed as a capstone project for the Senegal: Arts & Culture academic program of the School for International Training in Spring of 2005 under the direction of Souleye Diallo. The project was advised by Mamadou Aliou Diallo, Université Cheikh Anta Diop. It was published in the 2006 issue of Wittenberg University’s non-fiction literary magazine, Spectrum .
Courses Physics for Elementary Teachers An introductory course built around a student-oriented pedagogy where almost all the course concepts are developed through direct experimentation in groups. It focuses on building a conceptual understanding of energy and forces and using those concepts as a framework for understanding gravity, magnetism, electricity, and light. The course becomes…