Stop Trusting an AI That Never Disagrees With You

A reader asked me a good question this week. Not about ET rates or controller firmware. About AI itself.

The question: if an AI supports your view on sustainability, is it because you’re right, or because it’s built to agree with you? And if it can be tuned to agree, what stops it from quietly shaping how you think?

Fair question. Worth answering in public, because AI is now writing irrigation schedules, drafting water budgets, and advising utility boards. If it has a lean, the industry should know.

Split this into two categories.

Some things are settled. Human-caused climate change. The mechanics of evapotranspiration. The fact that overwatering turf doesn’t make it healthier. An AI system that pretends these are open debates, just to make a user feel validated, has failed at its basic job. That’s not neutrality. That’s flattery dressed up as balance.

Other things are not settled. Should a district price water by tier or by budget-based allocation? Is centralized smart control worth the cost over distributed soil sensors? Should a utility fund turf replacement or graywater systems with the same dollar amount? These are tradeoffs. Reasonable people in this industry disagree, and for real reasons: local hydrology, rate structures, politics, and capital constraints.

A good AI tool treats these two categories differently. Settled science gets a straight answer. Contested tradeoffs get the actual range of expert opinion, not the tool’s personal favorite.

Here’s what that looks like in practice. Ask an AI tool whether smart controllers reduce landscape water use, and a good answer is direct: yes, the field data is consistent on that, with a range depending on baseline scheduling quality. Ask the same tool whether a municipality should mandate smart controllers versus offering rebates, and a good answer stops being direct. Mandates get compliance faster but cost more in enforcement and pushback. Rebates get slower adoption but better buy-in and lower administrative overhead. Districts with tight staffing lean one way, districts with strong political capital lean the other. Neither is the “right” answer, and a tool that picks one and states it with the same confidence as the ET data has quietly swapped a policy opinion for a fact.

Nobody worried about AI thinks the danger is a robot cackling and feeding you false data. The danger is quieter. Models get trained to be helpful and pleasant. Pleasant slides into agreeable. Agreeable slides into telling you what you already believe, back to you, with more confidence than you started with.

For a water manager, that’s a specific and practical risk. If you ask an AI to evaluate your drought response plan, and it just validates the plan you already wrote, you haven’t gotten a second opinion. You’ve gotten an echo with better grammar.

Ask an AI company what prevents this, and you’ll usually get a policy statement. Here’s the more mechanical answer, and where it falls short.

Training data sets the starting bias, and nobody fully audits it. A model learns from a huge slice of human writing. That writing already contains skew, on climate, on politics, on water policy, on everything. Filtering catches some of it. Nobody inspects all of it. So the model starts from an inherited lean, not a blank slate. That’s a real, unresolved limitation, not a talking point.

Human feedback is where agreeableness gets baked in, or trained out. After initial training, companies use human reviewers to rate responses and reinforce the ones people prefer. This step is exactly where sycophancy creeps in, because “the answer I wanted to hear” often scores higher than “the answer that challenged me.” A company serious about the problem has to grade responses specifically for pushing back when warranted, not just for being pleasant. Most of the industry is still weak here.

Written rules can instruct a model to separate settled fact from contested opinion, and to argue against the user on request. This helps. It is still a soft constraint. A determined user can sometimes talk a model out of it through persistence or reframing, much like a persuasive person can wear down a colleague. Rules reduce the problem. They don’t eliminate it.

External review and red-teaming catch some of the drift after the fact. Researchers test models for political lean and excessive agreement. Real, but it’s sampling, not full coverage. It catches patterns, not every instance, on every topic.

None of this replaces the user. This is the part companies underplay. The most reliable check isn’t inside the model. It’s a user who notices a tool that only ever validates them and treats that as a warning sign rather than a compliment. Mechanisms within the model reduce the likelihood of drift. They don’t guarantee it won’t happen to you, on your question, today.

A few practical habits, whether you’re using AI for controller programming, rate design, board reports, or content like this newsletter:

Ask it to argue the other side. If you propose a plan, ask the tool to build the strongest case against it. A tool that can’t or won’t do this is not giving you real analysis.

Watch for constant agreement. If every idea you bring gets validated, that’s not a sign you’re right every time. It’s a sign the tool isn’t doing its job.

Keep settled science and open debate separate in your own head. Don’t let a tool blur “the data says X” with “reasonable people could go either way.” Push it to tell you which one it’s giving you.

Cross-check anything that drives a budget or a board decision. No AI tool should be the only source behind a six-figure capital recommendation.

Next week: the harder version of this question. Can the person running an AI company actually shift how the public thinks, on purpose, and get away with it? Short answer: yes, more than most people assume, and less invisibly than the fear implies. Part 2 walks through the evidence, including a real case that has already occurred, and applies the same scrutiny to the tool that writes this newsletter.

"Men are disturbed not by things, but by the view which they take of them." - Epictetus