Hey everyone,
On April 16th 2026, hundreds of data engineers gathered at The Contemporary Jewish Museum in San Francisco for the third annual Data Engineering Open Forum. DEOF 2026 was a day packed with technical depth, honest conversations, and the kind of energy you only get when a community shows up for each other.
We want to share what happened, what people took away, and what it all means for where data engineering is heading.
— Team DET 💛
tl;dr: 📼 Talk recordings out on YouTube
“DEOF is the professional highlight of my year.”
Yaakov Bressler (LinkedIn post)
“I’ve been to quite a few conferences, and this was one of the best so far. What made it stand out? It was purely community-driven.”
Yang (Eric) Liu (LinkedIn post)
“Data engineering is changing, not disappearing. Agents rely on pipelines just as much as business users do, and everything depends on strong, reliable data foundations.”
Michelle Winters (LinkedIn post)
“At petabyte-scale, architecture isn’t just about what’s technically great, it’s about the constraints the business puts on you.”
Narasimha G. (LinkedIn post)
“The role is evolving fast, especially around owning and operating agentic workflows, building in the right guardrails, and making sure observability and alerting are part of the picture from the start.”
Jordan Lewis (LinkedIn post)
“AI doesn’t replace data engineering. What will change is the scope of the role. It’s about thinking of data as a product, how it is used, and the impact it has on the business.”
Valentin Marek (LinkedIn post)
For a truly comprehensive recap, check out Pawel Mikler’s detailed write-up. Pawel holds the unofficial title of the only attendee who crossed the Atlantic from Europe specifically for DEOF, making it three years in a row.
Read Pawel’s full recap on LinkedIn
Pictures by DuHon Photography
This data engineering tutorial teaches you to build real-time IoT analytics using time-series datasets with relational metadata. You will learn to:
Optimize PostgreSQL tables into time-series hypertables.
Use compression to achieve 10x storage reduction and faster queries.
Enable lightning-fast analytics with pre-compute aggregation.
👉 Get started with the project here: Github.
(This post is sponsored by TigerData)
Our DET volunteers were embedded throughout the day, facilitating conversations, running the community booth, and soaking it all in. Here is what stood out to them.
Sanchit was struck by Dinesh Thangaraju’s talk on federated knowledge infrastructure:
“It reinforced fundamentals we’ve heard before, getting references right, avoiding multiple competing definitions for something as basic as ‘revenue’, but raised the stakes dramatically. In the AI age, a single wrong reference can cascade into serious output errors.”
His unexpected takeaway? AI adoption is rising from the bottom up, not the top down:
“I discovered use cases from talks that I hadn’t even considered, things that could be replicated at my organization immediately. We need a demo/showcase culture to surface this bottom-up innovation.”
Yaakov noticed data engineering branching into two diverging paths:
“People building coding agents, self-healing bots, context layers, to speed up DE execution. This role becomes a builder role and starts looking more like PM. Data Infrastructure, deeper technical capabilities. Building infrastructure for agents to execute on and with. This role starts evolving into a new kind of software/data builder.”
His conclusion? DE is changing fast, and the best way to keep up is to show up.
Indrajit Roy (Databricks) opened with the evolution of Apache Spark Structured Streaming and the introduction of Spark Declarative Pipelines. The takeaway: real-time capabilities are no longer reserved for streaming specialists. Express the logic, and the engine handles the rest.
Jerry Wang (Airbnb) followed with a 15-year retrospective on data infrastructure, making the case for the full-stack data engineer and the death of the “chain of custody” model.
The closing keynote panel brought together Laura Pruitt (Netflix), Paul Ellwood (OpenAI), and Vikram Koka (Astronomer), moderated by Michelle Winters. They tackled the hard questions: what does the future of data engineering look like? What skills should we double down on? How do we make sure AI truly serves humanity? An absolute must-watch for every data engineer!
Catch all DEOF talks in this YouTube playlist!
Xinran closed the day with words that many of us will carry for a long time:
“Change is scary, but so is staying the same.”
She asked the room to take one thing home: courage. The courage to start that project you have been putting off. The courage to reach out to someone new. The courage to evolve alongside this field rather than resisting the change.
“If you were inspired by a talk, do your follow-up research and try to adopt it in your own work. If you met a new friend, connect on LinkedIn and remember to catch up again later. If you felt courageous, start on that new project that you were afraid to pick up. When we meet again, let me know how it goes.”
Xinran Waibel (LinkedIn post)
Stay in the Loop
Join our Google Group for DEOF 2027 announcements, CFP updates, volunteer opportunities, and community news.
Sponsor DEOF 2027
Interested in partnering with the data engineering community? Complete our sponsorship interest form.
DEOF 2026 would not have happened without the collective effort of so many people.
Our speakers shared their knowledge generously. From keynotes to deep dives, every session reflected real experience and genuine insight.
Our program committee designed the programming and supported speakers to deliver their best: Apoorva Bapat, Goutham Budati, Jerry Wang, Michelle Winters, Sharath Chandra, Shruthi Jaganathan, Tulika Bhatt, Will Monge, and Xinran Waibel.
Our volunteer leads kept everything running smoothly: Anna Peng, Annu Joshi, and Balachandar Paulraj, along with the entire volunteer crew who facilitated conversations and helped every attendee get the most out of the day.
Our sponsors made this community event possible. We want to give a special thank you to Team Databricks (Denny and Lisa) and Team Astronomer (Caitlin and Volker) for their support and trust from the very beginning.
Our attendees showed up, asked real questions, shared lessons honestly, and stepped out of their comfort zones. You are the reason this community exists.
Data Engineer Things (DET) is a global community built by data engineers for data engineers. Subscribe to the newsletter and follow us on LinkedIn.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.