AI project management isn't software project management: 7 things that are different

Hello everyone,

If you have ever run a software project on time and on scope, then watched an AI project of the “same size” slide past every deadline while everyone insisted it was almost working, you already know the feeling we are naming today. In this article we are going to deep dive into why managing an AI project (classic machine learning and the newer GenAI/LLM kind) is a genuinely different job from managing a software project, broken into 7 concrete differences. Field notes, a worked estimation example, and a couple of kitchen analogies. Let’s go.

Managing AI like software is not a harmless style choice. It shows up in the failure rate.

In its 2025 Voice of the Enterprise survey of more than 1,000 organizations across North America and Europe, S&P Global Market Intelligence found the share of companies abandoning most of their AI initiatives before production jumped from 17% to 42% in a single year, with the average organization scrapping 46% of proof-of-concepts before they reached production. RAND (Aug 2024) put the AI project failure rate above 80%, roughly twice the rate of non-AI IT projects. And these are not, mostly, model-quality failures. They are planning, expectation, and process failures: the exact things project management is supposed to handle.

Here is the uncomfortable part. A lot of that failure comes from smart, experienced project managers doing what worked for fifteen years on software, and being quietly betrayed by it. The tools that made you good at shipping software are, in a few specific places, the tools that sink an AI project. Let’s see where.

Before the differences, let’s be fair to the overlap, because it is the source of the confusion.

Both software and AI projects have stakeholders, budgets, deadlines, scope, and a team you have to keep unblocked and sane. Both benefit from iterative delivery, short feedback loops, and shipping something small before shipping something big. If you are a strong software PM, maybe 70% of your instincts transfer perfectly. This article is about the other 30%, the part that does not transfer, because that 30% is where AI projects go to die.

ℹ️ Small glossary

• Deterministic = same input always produces the same output. A login form with correct credentials lets you in, every single time.

• Probabilistic = the output is a distribution, not a guarantee. The same input can produce different outputs, and “correct” is a rate, not a yes/no.

• Definition of Done (DoD) = the agreed checklist that says a piece of work is finished and shippable.

• Eval = an evaluation suite: a scored test of an AI system’s output quality, used where unit tests would be in software.

• Drift = the slow (or sudden) change in production data compared with the data the model was trained or tested on.

Here is the core claim, then we will unpack it. In software, you are managing the construction of a known thing. In AI, you are managing a search for a thing that might not exist. Same Gantt chart, completely different underlying activity.

A nuance worth stating up front, because it earns trust: none of this makes software PM “easy” or AI PM “harder.” Plenty of software projects are brutal, and plenty of AI projects are calm. The point is that the shape of the uncertainty is different, so the management moves have to be different. Now, the seven.

In software, a feature works or it doesn’t. The DoD is a checklist you can tick. In AI, the system is right most of the time, and “most” is a number you negotiate.

Think of two chefs. The software chef follows a recipe: same ingredients, same steps, same dish, every service. The AI chef is a tasting menu that is brilliant 92 times out of 100 and, on the other 8, sends out something nobody ordered. Your job as PM is no longer “is it done?” It is “what hit rate is good enough to ship, and who decides?” That single shift breaks acceptance criteria, QA sign-off, and the very idea of a bug. Is a wrong answer a defect to fix, or an expected 3% that you designed for? In software that question is nonsense. In AI it is the whole conversation.

⇒ Management move: replace “works / doesn’t work” with a target metric and a threshold agreed with stakeholders before the build, not after.

In a software project, unclear requirements are a failure you can fix by talking to stakeholders. In an AI project, some requirements are genuinely unknowable until you touch the data. You often cannot say whether an approach will hit the accuracy bar until you have tried it, which means part of your plan is a research bet, not a build task.

This is the difference between building a house from a blueprint and running an expedition to find a mountain pass. The blueprint is knowable in advance. The pass might not exist, and you only find out by walking. As one survey of the field puts it, in AI it can be unclear whether a given approach will yield the desired result until it is tried. A PM who promises a fixed spec on a discovery problem is writing a cheque the data may refuse to cash.

⇒ Management move: split the project explicitly into a feasibility phase (can this even work?) and a delivery phase (now let’s build it properly), with a real go/no-go gate between them.

In software, your raw material is code, and you control it. In AI, your raw material is data, and you usually don’t. It arrives late, dirty, mislabeled, biased, or subject to privacy rules nobody mentioned at kickoff.

There is a famous statistic that data scientists spend 80% of their time cleaning and preparing data. It is worth being honest here: that number is contested, it traces back to a small 2016 CrowdFlower survey, and later measurements from Anaconda put data preparation closer to 45% of the workday. But even the skeptics and the conservative surveys agree on the direction: data preparation is consistently the single largest time commitment in a data project, somewhere in a 25% to 80% range depending on what you count. For a software PM, that reframes the whole plan. The riskiest, longest, least visible task is not the model. It is getting the data into a usable state, and it lands at the start, where your stakeholders are least patient.

⇒ Management move: schedule data readiness as the biggest early task, with its own owner, and never let a stakeholder believe the model is where the time goes.

Software estimation is imperfect, but it rests on a fair assumption: the work is knowable and has been done before in some form. AI estimation often violates that assumption. “Improve accuracy from 82% to 90%” might take an afternoon or three months, and at kickoff you honestly cannot tell which, because you do not yet know if the signal is in the data.

Here is a tiny worked example of the same task estimated two ways:

⇒ Management move: estimate AI work in timeboxes, not deliverables. “We will spend two weeks finding out if 90% is reachable” is an honest estimate. “We will deliver 90% in two weeks” is a hope wearing a suit.

In software, you assert that output == expected. That assertion is the backbone of QA. In AI, and especially with LLMs, you often cannot assert exact equality, because the same input can produce slightly different output, even with the temperature dialed to zero, thanks to hardware numerics and provider-side batching.

So the test itself changes shape. Instead of unit tests, AI teams build evals: scored suites that rate outputs on dimensions like correctness, faithfulness, relevance, and safety, then let those scores gate a release. A practical split many teams use: fast automated metrics on 100% of outputs for regression testing, an LLM-as-judge on 10 to 20% of traffic for continuous monitoring, and human review on the 5 to 10% that is high-stakes. As one engineer put it, evals are becoming to AI what unit tests are to software. For a PM this is a line item you have probably never budgeted: building and maintaining the thing that decides whether the system is good.

⇒ Management move: treat “build the eval harness” as a real deliverable with its own time and owner, ideally before feature work, because if you cannot measure quality you cannot manage it.

Software has a comforting property: the ground holds still. Your dependencies change on a schedule you mostly control. In the GenAI world, the ground moves on someone else’s schedule. Your model provider can deprecate a model, change pricing, or ship a “better” version that silently shifts your outputs, sometimes with 30 days’ notice and no clean migration path.

Worse, “improving” your own prompt can quietly break cases that used to work, which is why teams now run regression evals on prompt changes the way software teams run them on code. The artifact you are version-controlling is no longer just code; it is a prompt template, a model version, and a retrieval index, all of which can drift independently.

⇒ Management move: pin model versions, keep a provider-swap plan (today’s GPT to tomorrow’s Claude or Gemini without rewriting the app), and treat every prompt change as a code change that must pass evals.

In most software projects, launch is the celebration. The bulk of the work is behind you, and maintenance is a smaller, calmer phase. In AI, launch is roughly halftime. The moment real users touch the system, drift starts, and the model that was excellent in testing begins, slowly, to decay as the world moves away from its training data.

This is the difference between building a bridge and adopting a garden. The bridge, once built, mostly stands. The garden needs watering, weeding, and replanting forever, and if you stop, it dies. That is why AI projects need a post-launch plan from day one: monitoring, a retraining trigger, and a named owner for model health. A software PM who plans for a launch-day finish line has under-scoped the project by about half.

⇒ Management move: budget the operations phase (monitoring, retraining, on-call, evals) as a first-class part of the project, not as “maintenance” you will figure out later.

Three quieter differences sit underneath the seven and compound them.

The team is more diverse: alongside developers and testers, you have data scientists, ML engineers, and data engineers, whose work is experimental and does not decompose neatly into two-week sprints. The risk register grows new entries software rarely sees: biased outputs, data privacy exposure, model drift, hallucination, and now regulatory obligations under frameworks like the EU AI Act. And stakeholder expectations are warped by hype: after a magical demo, stakeholders often expect 100% from a system that is designed to be right 92% of the time, so a large part of the PM job becomes expectation management, translating “probabilistic” into language a board will accept.

⇒ Management move: at kickoff, socialize the target metric (not “it works”), name the AI-specific risks out loud, and get the interdisciplinary team into shared rituals rather than forcing data science into a software sprint cadence.

Yes, and the framing matters more than any single tactic. Quick recap:

• “Done” is a threshold, not a checkbox. The core software instinct, works or doesn’t, breaks on a probabilistic system. Negotiate the hit rate before you build.

• Part of the plan is research, not construction. Some AI requirements are unknowable until you touch the data ⇒ separate a feasibility gate from delivery.

• Data is the long pole, and you don’t control it. Whether it is 45% or 80% of the effort, data readiness is the biggest, riskiest, least visible task, and it lands early.

• Estimate in timeboxes, not deliverables. “Two weeks to find out if 90% is reachable” is honest. “90% in two weeks” is a hope.

• Evals replace unit tests, and someone has to build them. If you cannot measure quality, you cannot manage it. Budget the harness.

• Launch is halftime. Drift starts when users arrive, so monitoring, retraining, and an owner are day-one scope, not maintenance.

The one-sentence version: software project management asks “are we building the thing right?”, and AI project management has to keep asking, all the way through, “does the thing even exist, and is it still true today?”

✅ The 5-minute self-diagnosis

One honest yes/no per row:

• Have you agreed a target metric and threshold with stakeholders, instead of “it works”?

• Is there a real feasibility gate before the delivery commitment?

• Is data readiness scheduled as the biggest early task, with its own owner?

• Did you budget building and maintaining an eval harness?

• Is there a post-launch plan (monitoring, retraining trigger, model owner) already scoped?

Three or more “no”s? You are running an AI project on a software plan.

Which of these seven has already bitten you?
Hit reply and tell me the one that cost you the most. I read every response.

Research / standards

• RAND (Aug 2024). The Root Causes of Failure for Artificial Intelligence Projects. https://www.rand.org/pubs/research_reports/RRA2680-1.html

• Wirth, R. & Hipp, J. (2000). CRISP-DM: Towards a Standard Process Model for Data Mining. (Reference framework for the iterative ML lifecycle.) https://www.datascience-pm.com/crisp-dm-2/

• When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications (arXiv, 2026). https://arxiv.org/html/2601.22025

Industry / news

• S&P Global Market Intelligence (2025). Generative AI shows rapid growth but yields mixed results (17% → 42% abandonment; 46% of POCs scrapped). https://www.spglobal.com/market-intelligence/en/news-insights/research/2025/10/generative-ai-shows-rapid-growth-but-yields-mixed-results

• The Pragmatic Engineer (2025). A pragmatic guide to LLM evals for devs.

• Applied AI (2025). The LLM Application Lifecycle: From Prompt to Production. https://www.applied-ai.com/briefings/llm-application-lifecycle/

• DAC.digital (2026). 5 Key Differences Between Running an AI Project vs Software Project. https://dac.digital/ai-project-vs-software-project-management/

• Amperity (2026). The cost of poor data quality (data-prep time range, 25–80%, Anaconda ~45%). https://amperity.com/blog/cost-of-poor-data-quality

• Forbes / Gil Press (2016). Cleaning Big Data: Most Time-Consuming, Least Enjoyable Data Science Task (origin of the 80% stat). https://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/