Your Next 'AI Engineer' Isn’t One Today

As an active investor in applied AI companies (among other things), I run into a ton of startups on the hunt for “AI engineers.” It’s become one of those roles that everyone says they need, but nobody seems to define it the same way.

Gergely Orosz from The Pragmatic Engineer interviewed Chip Huyen who defines AI in her book “AI Engineering”. In my opinion her definition is spot on: Building applications that use LLMs, placing it between software engineering (building software products using languages, frameworks, and APIs), and ML engineering (building models for applications to use). According to her, AI engineering feels closer to software engineering because it usually starts with software engineers building AI applications using LLM APIs. The more complex the AI engineering use case, the more it can morph into looking like ML engineering.

Like any buzzword, there’s a lot of nuance hiding behind the title. Recently, I was helping a few of my portfolio companies with recruiting for this role. While running the ops side of talent pooling and outreach to talent, I realized something: AI engineers can mean very different things depending on the type of company and what they actually need and AI engineers have diverse backgrounds.

One founder’s AI engineer is another founder’s data scientist, or software engineer, or even ML engineer/researcher.

Here’s the kicker: your next “AI engineer” probably isn’t working as an AI engineer right now.

As demand for AI engineers soars, you won’t find top talent by just scrolling through LinkedIn profiles of folks that are already working as AI engineers.

So the real question is: where do you actually find them, and what roles are they sitting in today?

To get a clearer picture, I went through 50+ AI engineer profiles from venture-backed startups. The goal was to spot patterns in backgrounds so I could help teams be more precise when searching for talent.

Here is what I found:

So the key finding is that AI engineers mainly come from 3 backgrounds, almost evenly distributed: Data Science, Software Engineering, and Machine Learning Engineering.

Which path makes the most sense really depends on what the startup is building and where the team needs support. Often, a company will hire people from different backgrounds, with each one focusing on different aspects of the AI work.

In the early days, I would lean toward hiring generalists who can wear multiple hats. I’ve also noticed (though I haven’t quantified it yet) that the more technically complex the product, the more companies favor senior ML engineers over data scientists or software engineers, which intuitively makes sense. (rule of thumb: if you have to train a model, get a ML engineer)

Another hunch of mine is that there are basically 2 types of Data Scientists, either with technical or with a business background. Without having the numbers to back it up I would argue that only for very thin model wrapper startups, data scientists without a solid technical background are promising hires.

In general, I like to think of matching backgrounds to hiring profiles as flexible, not black-and-white. People often have adjacent strengths beyond their core experience, so I mentally visualize each profile as a spider diagram to capture their range of capabilities (e.g. Software Engineer with strong LLM experience or a Data Scientist with data engineering chops).

Overview of technical jobs mapped to backgrounds at a glance:

Model Adaptation & Fine-Tuning

• Adapt/fine-tune pre-trained models; optimize for speed and memory usage.

—> ML Engineer

Prompt Engineering

• Design and optimize prompts for LLMs; reduce hallucinations; improve controllability and consistency.

—> Data Scientist or Software Engineer with LLM experience

Rapid Prototyping

• Build quick POCs; benchmark models/prompts; iterate with user feedback.

—> Data Scientist or Software Engineer

Data Engineering & Pipeline Management

• Design scalable pipelines; clean and preprocess data; ensure data quality; maintain embeddings for LLMs.

—> likely dedicated Data Engineer if you are building with robustness and scalability in mind

Knowledge Management / RAG

• Integrate vector search with LLMs; manage indexing and retrieval; ensure relevant, factual, context-rich outputs.

—> Software Engineer with data engineering experience and some ML familiarity

Model/Feature Evaluation

• Define metrics (accuracy, latency, fairness); test edge cases; incorporate human feedback.

—> Data Scientist

Monitoring in Production

• Track drift, errors, usage; design dashboards for ongoing reliability.

—> ML Engineer or Software Engineer with MLOps experience

Deployment & DevOps

• Package models into services/APIs; manage CI/CD; optimize for production reliability and scale.

—> ML Engineer or Software Engineer with ML experience

As a founder, I would start by mapping the product’s needs and spotting capability gaps in the team. That drives the hiring profiles I build. Early on, some roles can be handled by generalists, think software engineers with LLM experience. As you grow, these hybrid roles split into specialists.