0x19: How Not to Suck at AI

Too many initiatives start with “AI all the things” rather than a more surgical approach. Businesses are complicated patchwork quilts of processes and systems, built over years and decades. Contrary to the hype being shoveled to boards and executives, dropping generative AI into the business, as the MIT Project NANDA report suggests, isn't a silver bullet and failure is seemingly guaranteed. So, let’s explore why.

A Note From Me!

I’ve been so busy learning and playing that I haven’t been writing/recording much. This essay was drafted back in September and recorded in December, but never edited the video. I felt it was getting stale, so I’m publishing it as is to focus on something cool. Stay tuned!

Now back to the essay.

One of the problems I see related to the MIT report is that companies don’t know how to actually innovate. Sure, funds are allocated to buy something shining and tinker with it for a few months under the guise of innovation. However, for most companies, their risk appetite for failure is near zero and their patience for an outcome is typically measured in months and quarters; certainly not years. That isn’t innovation, that is continuous improvement.

Proper innovation, creating a “sea change” with how a business functions, is hard for most companies to fathom. That is because innovation is not about buying the new whiz-bang, but rather challenging the status quo and implementing a solution that currently does not exist. Innovation is about taking risks and accepting the likelihood for success is low, but doing so anyways. Because companies who truly innovate and take on such risks, do so because they know the potential will be… HUGE!

Following on my suggestion of centralizing innovation within a crack team, over the de-centralize and finding out strategy, the process of innovating continuously improving within a business, AI or not, is about understanding what problem(s) are we trying to solve and leveraging people, processes and technologies to do so.

I suspect this overly common failure cycle of technology projects is rooted in our inability to fully comprehend the complexity of the problem we are trying to solve as well as our failure to articulate achievements before the business’ appetite wanes. We always start off with good intentions, but are eventually overwhelmed with the unknown and/or scope creep that blurs our success criteria and eventually leads to (yet another) failed project.

Prior to the hype of generative AI, Low Code/No Code (LC/NC) and Business Process Automation (BPA) solutions were the last hyped savior of business productivity debt. And like most novel technologies, they brought some serious potential, but fell into the “trough of disillusionment” because companies belly flopped the people and process side of the equation.

What’s old is now new again and the challenges that LC/NC and BPA experienced are here again with generative AI. Both were sold as silver bullet solutions to what ails businesses, but to repeat history, have fallen flat in meeting the transformative expectations. I suspect a lot of these challenges have to do with businesses wanting to fix their complex and failure prone processes and systems, and the newest Whiz-Bang promising they can do so, for just a bit more money.

The reality is, there are no solutions out there that can ingest complexity and output simplicity (well), at least not on its own. Put down the ChatGPT!

Part of the reason for this is that these complex systems were built by a far more complex and more so irrational system: humans, with all our social and cognitive flaws. Thus, whether it be LC/NC or generative AI, successful innovation continuous improvement requires a deeper understanding of the problem we are solving and more importantly understanding the atomic parts of the process or system.

Yes we are splitting atoms, which seems to contradict what all these solutions seems to be selling: an all encompassing system to fuse your business into a highly automated juggernaut. But the reality is and the MIT report backs this up, AI is not the silver bullet that can infer the nuances we have built over the decades, into a solution that just works.

Instead, we need to dissect our processes to understand them at the atomic level so that we can leverage our tools, whether it been LC/NC, BPA or the current hype of agentic AI to, actually move the needle forward.

It took a while, but my wife finally gets me and my hyper logical if-then-else mindset. I think in flow charts, and I have thrived with NC/LC, generative AI and now agentic AI, with pretty great results... most of the time. I have to thank my 14 year old self in the 90’s for buying Visual Basic (at Future Shop!) and learning to code over a summer. While I ended up not going down the computer science path, those early days of coding, tinkering and hacking gave me an understanding of how computers work, how to build complex systems and most importantly, how and why complex systems fail.

So, as it relates to leveraging AI, it is important to understand how generative AI works, its limitations and how to maximize the accuracy, consistency and transparency of its outputs. At a foundational level generative AI is a probabilistic word soup generator: closer to a rambling idiot, than an all knowing oracle. For normal people this means: AI has read everything we ever wrote for the last 2-3000 years and statistically speaking can predict the most likely next word. It is more nuanced than that, but that simplification I hope can convince you it isn’t intelligent, at least not on its own. I usually describe LLMs as having an incredible understanding of the structure of our language and how we like to communicate (verbal and more importantly written), but it doesn’t understand what we are talking about.

So to truly be successful with AI adoption, is to know the limitations of AI and know how to use its strengths to your favor. For the most part this is about understanding the problem and breaking the process up into atomic parts, before you start to solve.

As mentioned, I learned how to code in Visual Basic at home, learned Pascal in high school, C in College, and enough to be dangerous with python to get a simple project done. But by no means am I a developer and before the introduction of LC/NC, BPA and now AI, was limited in my ability to create and innovate.

AI has allowed us the luxury of natural language processing, over learning to code, but it doesn’t stop there. Prompt engineering AKA knowing how to talk computers, is the difference between beautiful but mediocre outputs and something you are proud of and actually works. Prompt engineering is all about guiding the LLM with highly descriptive guardrails to maximize ideal outputs while minimizing undesirable hallucinations and inconsistencies. There is a lot more I have learned on this topic, and the new hotness is context engineering, but needless to say if your prompts are smaller than a tweet, your mileage is going vary.