Ian Whitestone

Extend the Runway: A deep dive into Snowflake costs at Coalesce 2022

Calculating cost per query in Snowflake

Snowflake Performance Tuning and Cost Optimization

Snowflake Optimization Power Hour

Snowflake Architecture Overview

What’s up with DuckDB?

Farewell, Shopify ❤️

👋

Unpacking the Spark Web UI

--> Example Job & Data Navigating the UI Jobs Stages SQL Plans Storage, Environment and Executors Notes Generating the dataset Simulating Skewness Transaction Dataset Shop Dimension Dataset The Spark Web UI provides an interface for users to monitor and inspect details of their Spark application. You can leverage it to answer a host of questions like: How long did my job take to run? How did the…

Spark from 100ft

--> Architecture Overview & Common Terminology Example 1: Aggregating transaction amounts by app Sample Code Execution Overview Stage 1 Shuffle + Stage 2 Example 2: Enrich a set of user events in a particular timeframe Sample Code Execution Overview Notes Apache Spark is an open-source framework for large-scale data analytics. Large-scale data processing is achieved by leveraging a cluster of…

In the trenches with Spark