Data Engineer Things Newsletter - Community Spotlight Edition (Apr 2026)

Hello, everyone!

This edition of Community Spotlight features Michelle Winters, a self-proclaimed data nerd who has built petabyte-scale platform tooling for Netflix, architected diverse data systems at eBay, and founded a highly successful AI-tooling startup.

I had the great fortune of sitting down for a boba chat with Michelle to distill the architectural mental models that hold up at massive scale. We also explored the convergence of data engineering and AI, and why she believes the engineer’s role is shifting from simply moving pipelines to actively stewarding data meaning.

- Sugandhi

We are hosting the third Data Engineering Open Forum (DEOF) on April 16 in San Francisco! Here is what you will experience expect at the event:

• Sessions by speakers who are solving cutting edge problems in data engineering, for examples, Apache project creators and PMCs like Julien Le Dem, Boyang Jerry Peng, and Jack Ye.

• Intentionally-designed activities that you can sign up to engage in small-group networking (because we know it’s hard to make conversations at conference).

• Opportunities to connect with data engineering teams at top tech companies (like Airbnb, Netflix, OpenAI, and more) at their booths.

Our ultimate goal is that when you look back some day, you could confidently say “I’m so glad I went to DEOF”, because the people you met there or ideas you walked away with made a difference in your career.

👉 See the agenda HERE. RSVP today!

“Moving data alone doesn't create much value, and most of this work will be automated in the near future anyway. The real role of a data engineer is to ensure trust and preserve meaning.”

For those in the DET community who may not know you yet, could you briefly introduce yourself?

Hello, everyone—my name’s Michelle. I’m a lifelong data nerd who’s done a bit of everything in the world of data. Most of my work has focused on data engineering and warehousing, data platform and infrastructure, system intelligence, automation, efficiency, scalability, quality, and monetization. I’ve been fortunate to work at amazing companies like Netflix and eBay, and I even founded my own startup which provided collaborative data tools and became a top 10 plugin for OpenAI’s marketplace before it was acquired. Over the years, I’ve worn many hats—data engineer, architect, leader, founder—but my favorite hat is simply “data nerd.” I still love exploring new ways to structure, understand, and leverage data to create real impact. I’m especially excited about the future of data when it comes to collaboration, innovation, and solving meaningful, real-world problems.

You’ve led data engineering at GoDaddy, built platform tooling for Netflix’s 100 petabyte warehouse, co-founded a startup, and architected data systems at eBay. Each of those environments is radically different in scale, constraints, and culture. Is there a single architectural principle or mental model that has held up across all of them?

If I had to distill it down to one idea, it’s this: design for the full 100% from day one but don’t treat all 100% equally.

About 80% of what people need should be simple, obvious, and well-paved. That’s your “highway.” It’s where most users should live, and it should feel easy, reliable, and almost boring in the very best way! That’s where you invest in strong defaults, automation, and clear observability so people can move quickly with confidence.

The remaining 20% is just as important, but it’s different. That’s your off-road terrain. This is where flexibility matters. You’re giving people room to explore, to handle edge cases, and to push boundaries without breaking the system. If you over-optimize for simplicity, you box people in. If you over-optimize for flexibility, you create chaos. The real craft is in holding both at the same time.

What this forces you to do is build the right abstractions. Not just tools that work, but systems that people can understand, extend, and truly own. And ownership is the goal, because that’s what unlocks speed and innovation at scale.

The other lesson that’s stuck with me is to resist building for a single use case, even if it’s the loudest one in the room. Early on at Netflix, we intentionally designed for three very different workloads: personalization, which was deeply custom, hand-crafted scalable Java code; content analytics, which leaned heavily on SQL; and finance, which had strict accuracy and audit requirements. Those use cases pulled the system in completely different directions, and that was the point. If you can design something that works well for very different kinds of users and constraints, you’re much more likely to end up with a platform that lasts.

For folks early in their careers, I’d frame it this way: don’t just solve the problem in front of you—zoom out and ask what kind of problems this solution should be able to handle a year from now. Be intentional early. Not just about what you’re building, but about how you’re building it and what it might need to become. The earlier you start thinking that way, the better your instincts will become.

What is one architectural principle that surprised you by not holding up?

For a long time, I believed you could separate the platform from the meaning of the data—that our job was simply to move data reliably from point A to point B.

I don’t think that holds up anymore. Moving data alone doesn’t create much value, and most of this work will be automated in the near future anyway.

The real role of a data engineer is to ensure trust and preserve meaning, making sure data carries its context all the way through to feature engineering, AI models, and ultimately decisions. That means stepping back and asking harder questions: Can we trust the data? Are we capturing what actually matters? Are we keeping too much? Are we filtering or summarizing low-value data early enough?

This is where the role becomes much more impactful. You’re not just building pipelines, you’re shaping how the business understands its data. So don’t stop at “does this pipeline work?” Ask what the data means and how it creates value. That shift in thinking is what sets great data engineers apart.

The industry talks a lot about the “full-stack data engineer” as roles between data engineering, data science, and ML engineering continue to blur. Is that convergence a good thing, or are we at risk of undervaluing what people with deep, specific expertise bring to the table?

Some of the best outcomes I’ve seen come from teams where people bring different strengths and perspectives. You need both breadth and depth: generalists can connect dots across systems, but specialists provide the deep domain knowledge and technical rigor that prevent costly mistakes. There isn’t a single “full-stack” role that fits every company or every problem. What matters is understanding where the organization’s core challenges lie and building complementary teams to address them.

That’s why I think the concept of a “data artisan” is so useful. These are people who combine technical skill with deep domain expertise, whether that’s advertising, healthcare, or finance. Their knowledge adds disproportionate value because they understand the nuances of the data and the context of its use. Looking ahead, I expect we’ll see people self-select into hybrid roles that blend technical skill and domain knowledge in ways we don’t yet have names for, and those roles will be increasingly critical as data-driven products grow more complex.

What is a skill that was considered “not a data engineer’s job” a few years ago that you now think is essential?

Anything related to AI has quickly moved from “nice to have” to table stakes.

At a minimum, data engineers should understand concepts like retrieval-augmented generation (RAG) and vectorization—how data is embedded, retrieved, and used in modern AI systems. These aren’t just modeling concerns anymore; they directly shape how data needs to be stored, indexed, and served.

We’re also seeing a shift with graph technologies. Historically, that was the domain of specialists, but the ongoing responsibility of building, populating, and maintaining graphs fits naturally with data engineering. As systems become more relationship-driven, that ownership is starting to move closer to the data layer.

👉 Read our interview with the founder of PuppyGraph to learn more about graph technologies.

Data quality is evolving too. It’s no longer just binary checks or simple rules. Increasingly, it’s probabilistic and algorithmic—designed to catch real issues without overwhelming teams with false positives. That shift brings data engineering closer to disciplines like statistics and machine learning.

What all of this points to is a broader trend: the boundaries are moving. Data engineers are no longer just responsible for pipelines, they’re becoming stewards of how data is structured, interpreted, and trusted in increasingly intelligent systems.

For those earlier in their careers, this is a great moment to lean in. You don’t need to become an ML expert overnight, but building intuition around these areas, especially how data flows into and supports AI will set you apart.

There is a lot of pressure right now to embed AI into data pipelines: agentic orchestration, LLM-based data quality checks, AI-generated transformations. Where is AI genuinely changing what a data engineer does day to day, and where is the hype running ahead of reality?

A lot of what we’re seeing right now feels premature, and the reason is simple: we still have a data problem.

In most organizations, data is disjointed, inconsistent, redundant, and poorly understood. And yet, we’re expecting AI to step in and magically extract meaning from it? That’s wishful thinking. If meaning doesn’t already exist, if it’s not encoded somewhere in the system, AI isn’t going to manufacture it for us. At least, not to a level we should have confidence in.

Where I do see real value is in AI acting as a collaborator. AI agents can generate artifacts, transformations, documentation, even quality checks, but those outputs still need expert review. If that slows things down a bit, that’s okay. Speed without confidence in the data is a liability, not an advantage.

At the end of the day, the fundamentals haven’t changed. If you don’t trust your underlying data, nothing built on top of it will hold up, no matter how advanced the AI is. Any organization that wants to compete in this space needs to take that seriously. That means investing in data quality, building systems that are understandable and observable and explainable, and treating security and consumer privacy as first-class concerns, not an afterthought.

You built Noteable from the ground up as CEO, and then stepped into a Distinguished Architect role at eBay. What did running a startup change about how you read a large organisation’s data architecture?

Running a startup made me think much more about culture—both team and company—and gave me a stronger sense of ownership and connection to the business, which I think makes me a better partner overall. But the biggest shift for me was humility. As CEO, I couldn’t be the most technical person in the room anymore, and getting comfortable with that changed how I lead. I learned that when you give people clear context and direction, they often take ideas further than you could yourself. That experience reshaped how I think about both systems and teams: the goal isn’t to have all the answers, but to create an environment where talented people can do their best work. When you invest in good people, give them ownership, and get out of their way, I’ve found they’ll consistently exceed your expectations.

eBay is a marketplace with an enormous variety of data: buyer behaviour, seller inventory, pricing, fraud, logistics. How does that diversity shape architectural decisions differently from a single-product company like Netflix?

Every company has a wide variety of diverse data—it’s really a question of proportions. What matters more than the domains themselves is the company’s unique data profile: where the data comes from, what latency it requires, and what happens if it doesn’t arrive on time. That’s why I’m not a fan of one-size-fits-all architectures. The better approach is to understand where your center of gravity is and design systems that minimize unnecessary movement while supporting those core needs. At the same time, you have to build in flexibility from the start. Even if you don’t have something like IoT data today, that can change quickly, and your architecture needs to be ready to evolve with it rather than be rebuilt from scratch.

As a Distinguished Architect, you are often shaping decisions without direct authority over the teams implementing them. How do you actually get things to change at that level?

You lead with the business problem: what are we trying to solve, and what is the customer impact? Put together a long-term strategy and focus on aligning people around the outcomes it would unlock. Make it tangible, then work backward from there. The challenge is that when teams are overworked, their capacity often only reaches the current sprint or next quarter. So you start with those who still have passion and headspace, get them in the trenches building proof points, and then bring everyone else along when you can demonstrate tangible value. Change happens person-to-person—you’re not selling a technical solution, you’re selling what the organization will be able to do tomorrow that it cannot do today.

What is one tough lesson from your career that made you a better data professional?

Protect your brain! Data professionals are often pushed to deliver under sustained pressure, and it’s easy to treat mental capacity as an unlimited resource. The reality is that cognitive performance degrades long before most people notice it, and by the time the signs are obvious, the cost is already high. Don’t wait until that point—invest in yourself early. When you take care of your mental and physical well-being, you may work fewer hours, but you’re more engaged, generate better ideas, see solutions that were previously hidden, and rediscover your passion. That matters enormously when the work demands genuine innovation.

• Architecture should be designed for both simplicity and flexibility from the start. Simplicity is the 80% path; the remaining 20% is where you design for flexibility. Use contrasting use cases to stress-test your platform before it needs to scale.

• The data engineer’s role is evolving beyond moving data. Full ownership of data meaning, from ingestion through to feature engineering and AI models, is where the real value lies. AI does not fix a data quality problem, it amplifies it.

• The future of data engineering is specialization by domain, not just by technical function. Industry knowledge will become as important as technical skill, and roles that do not yet have names are already beginning to emerge.

• Influence without authority starts with business outcomes, not technical vision. Find the believers first, build proof points with them, and let the results do the convincing.

• Cognitive capacity is finite. Sustained pressure degrades mental performance before most people notice, and by the time the signs are visible, the cost is already high. Investing in yourself is foundational to a long and effective career in data.

• LinkedIn

Data Engineer Things (DET) is a global community built by data engineers for data engineers. Subscribe to the newsletter and follow us on LinkedIn to gain access to exclusive learning resources and networking opportunities, including articles, webinars, meetups, conferences, mentorship, and much more.