Context Engineering for Intelligent Systems with Jamie Bono

Listen to “Context Engineering for Intelligent Systems with Jamie Bono” below, or on Apple Podcasts, Spotify, YouTube, Amazon, Pandora, or wherever you get your podcasts. The episode transcript is shared below the embed.

Frank: Welcome to TechXY Turbo, a tech podcast providing independent technology content and perspectives. My name is Frank Gullo and I am your host. On this episode, we’re joined by Jamie Bono, a principal consultant at EmergenceTek Group, where he designs information architectures and leads data platform strategy across healthcare, environmental services, and financial sectors.

Before EmergenceTeK, Jamie spent nearly a decade as a paramedic in the Buffalo Niagara region, then studied English at the University of Buffalo and at the University of Pittsburgh, where he was also a Homeland Security Research Fellow. Jamie went on to lead data integration and analytics for Medicaid redesign and value-based payment initiatives, then built NLP pipelines and knowledge graphs as a senior engineer at SUNY’s Department of Biomedical Informatics. Today, Jamie’s work centers on context engineering — structuring the information around systems so that people, teams, and AI agents working inside them can actually reason well. Jamie also serves on the board of CCNY in Buffalo, New York.

Jamie, we’ve been talking about this for a while. I’m really excited to have this conversation. How are you doing today?

Jamie: I’m doing great, Frank. Thanks a lot for having me on. I’m really excited.

Frank: Absolutely. That intro was a lot, and I really want to dig into how one gets from paramedic to data engineer. So let’s start from the beginning. Your beginning is genuinely unlike most people I’ve had on the show or even met — you were a paramedic for nearly a decade. How does someone go from running calls in Buffalo Niagara to building pipelines and data infrastructure? What connected this for you?

Jamie: That’s a really interesting question. When I started volunteering on the ambulance in high school, it was the only place where I could access a computer — that was an unexpected part of it. But as a consultant, the skills are really very similar. It’s about being able to work with people, being able to apply a triage instinct — to enter a situation where the stakes are really high for the people there and help identify reasonable solutions to immediate problems. It’s a lot like walking into a client’s data environment. You have to be useful fast. You don’t always have all of the information, and the stakes are really high. I think it’s more similar than you would think.

Frank: So anyone out there who’s a paramedic looking at different careers — data engineering may be one. Jamie, you’ve mentioned watching patients cycle back for the same preventable problems: housing instability, food insecurity, transportation. At what point did you decide the answer to these problems wasn’t necessarily better emergency response, but better data? And what did that shift feel like?

Jamie: To be honest, when I was working on the ambulance, I didn’t see these as larger social problems. I saw them through the lens of what we understood in the moment — part of what felt like the law of emergency services. It looked like people were abusing the system. It looked like they were calling 911 to get seen in the emergency room faster, or because they hadn’t planned well enough to get a primary care appointment. When someone has the flu and calls 911 at 2:00 in the morning, a lot of times as a paramedic it just looks like poor planning.

But when I got access to data as a specialist working on Medicaid redesign, those were the questions I wanted answers to — because that’s what I knew. I went looking for things like, is there evidence that people call 911 and then when they get to the hospital don’t go into the ER? An ambulance bill with no ER bill. And I found that the reality in the data didn’t really bear out the stories I had been hearing.

What I did see were these larger issues — access to care, barriers to care. A lot of times people aren’t noncompliant with their meds because they don’t want to take them. It’s that they have no way to get to the pharmacy. And I was really lucky to work with people with long histories in case management, behavioral health, primary care, housing security, and food access — and to see how these things I was observing with a different set of tools changed the way the problems looked. They looked very different than they did from the back of an ambulance.

Frank: That’s fascinating. And I think it’s really interesting that this all came before your data engineering work — when you were a paramedic, you had a very small part of the data picture, and it sounds like you’ve zoomed out and now have a much larger context. We’re going to talk a lot about context and AI systems, and I wonder how many people in their normal roles have just a piece of the data, and as you zoom out you get a bigger picture.

But back to your path — you went through disaster planning, Medicaid redesign, grad school for technical communication, biomedical informatics at UB. That’s a very non-obvious sequence. Looking back, does it all come together? Is there a pattern to it?

Jamie: Very different answers at the time versus in retrospect. At the time, I was really excited to solve complicated problems that seemed exciting. I also liked having a high degree of autonomy — as a paramedic, being able to make decisions in the moment and have ownership of those decisions. As a student, that same kind of agency.

And in both, I taught a lot. I really enjoyed teaching new paramedics, being a field training officer, even on scenes teaching bystanders and family why I was doing the things I was doing — because it made patient care easier and more effective. And I really liked reading. When you’re not busy in the ambulance, you just read.

Now when I look back, one aspect is that it’s really nice to feel needed. As a paramedic you feel needed. As a consultant you feel needed. But I think really, all of it is about how information is useful in the moment — how it flows through systems and where it gets jammed up. How do we communicate things more clearly? How do we plan to respond in ways that we can teach other people?

I think about CPR a lot as an example. It takes this huge body of clinical knowledge — physiology, chemistry, all these things — and distills it down to a set of steps a 12-year-old can execute in an emergency. It distills them down into very simple things that are shaped for the context.

Frank: Excellent. So everyone — make sure you know where your defibrillator is in your office.

Jamie: Yeah. At the end of the day: check for responsiveness, airway, breathing, circulation, do compressions, call 911. It becomes repeatable.

Frank: And that’s a good segue into context engineering and AI. You use a phrase I want to dig into — context engineering for intelligent systems. For listeners who think of AI mainly as a chatbot or a code assistant, explain what this phrase means and why you use it.

Jamie: Great question. I want to start with the formal definition, because I can’t take credit for it. It’s only been around about a year. The CEO of Shopify posted about context engineering on X as a broader category that prompt engineering fits within. Prompt engineering being about how you ask questions and instruct an LLM to respond. He said that context engineering is about providing the context the LLM uses to reason and make decisions — and I’m using the term “reasoning” pretty loosely.

Andrej Karpathy narrowed it down a bit further, saying it’s about shaping that context and making sure it fits in the window — making sure the AI has access to it. And now we’re reading a lot about how it’s the process of creating the set of rules that help the model perform the way you want it to perform.

I take it a little bigger, because I did spend a long time in an English program studying rhetoric and technical communication — how we write plans and protocols. And I really thought it wasn’t that novel. It’s something we’ve been doing in rhetoric and technical communication forever: how do we decide what the right information is to give people so they can make decisions in the moment? The CPR example again — a 12-year-old doesn’t need to know the history of cardiology. They just need to know what to look for and how to respond, because that’s what’s needed in the moment.

I like the framing of context engineering for intelligent systems because LLMs don’t operate alone. Humans are part of that system. Humans together form really complicated intelligence systems. So how do we provide those actors with the tools and information they need to make sound and reasonable decisions?

The chatbot trap a lot of people run into is that when you’re chatting with an LLM, the context is getting generated on the fly — what we call one-shotting. You ask a question, get a response, say “no, do it like this,” and it becomes this back and forth that is really inefficient. I think it’s because a lot of people are using it the way they use Google — you type something into a box and get a response. When we design systems, we look at things like instructions, system prompts, templates. Those are all part of the context that frames the conversation. It makes things a lot more efficient.

Frank: And that matters if the AI is only as good as the context you give it. So for the average business owner who’s just been handed a new paid account — whether it’s Copilot or Claude — should they feed it everything in their department folder, or be selective? What’s your advice for new business users coming into these tools?

Jamie: I would ask a complementary question: how would you onboard a new employee? If you’re onboarding a new junior team member, you don’t necessarily start by giving them the entire history of the company. That has a purpose, but it doesn’t help them fulfill their immediate goals. I think of onboarding large language models very similarly — what does it need to know to complete the tasks I want it to complete within my specifications? That doesn’t mean I don’t enjoy giving systems broader instructions to see what they can do. But it’s a tool. The question is: how do I onboard it onto the team? Not use it as a firehose for my thought process. Part of my expertise is being able to bring in the information that’s necessary to solve the current challenge.

Frank: Related — a lot of people I speak to focus on the model itself. Which one, which version, which prompt. You seem to focus more on the surrounding structure: knowledge graphs, information architecture, the layers. Why do you focus on that surrounding structure as much as — and more than most people focus on — the model itself?

Jamie: The models change super quickly. What is a state-of-the-art frontier model now is frontier for weeks. The development cycle has been compressed a lot. So focusing on the model is focusing on one tool, and that thing changes. Things like ontologies, knowledge graphs, ways of encoding information — and I’ll extend this to less exciting areas like data governance — these work across systems. Not just one model, but multiple models, a mixture of expert systems, human-in-the-loop systems. Context generalizes really well and scales to lots of different systems, where a model is just one tool.

That doesn’t mean certain approaches to context engineering don’t work better with certain models — there is clearly a linkage. But the work done in information architecture, knowledge engineering, and context engineering pays off in larger dividends than something applicable to just one model. It’s like learning English. If I learn English, it unlocks lots of different things, not just being able to speak English.

Frank: Makes sense. So you build and run AI systems regularly, and there’s a lot of conversation about agents right now. What does a well-designed agent system look like from a practitioner perspective day to day? And what does a poorly designed one look like as it starts to break down?

Jamie: A well-designed one is invisible. It should fade into the background and reduce friction. A lot of what I get asked to do as a consultant or data engineer is to design systems that address areas of high friction — business processes that are time-consuming or resource-consuming, places where automation is genuinely valuable. The best-designed systems disappear. There’s a saying: the best design is 99% invisible. I really think that’s true.

Poorly designed systems are really visible. They usually look fairly similar — they don’t have guardrails. They have a lot of description of what you want the system to do, but not a lot of description of failure modes, anti-patterns, or what not to do. Without guardrails, you get beautiful garbage. You get this proliferation of things that are close to accurate but aren’t always accurate — they perform accuracy really well. And a lot of it comes down to automating things we don’t understand yet. Trying to automate a process you haven’t spent real effort unpacking and understanding. That’s again a lot of what I do as a consultant — talking to people about why they make the choices they do and how they make them, in order to make sure we can give proper context to an agent system.

Frank: Do you think there’s been an agent tipping moment yet? Like — I remember when Napster seemed to really change how people thought of what was possible with the internet. I feel like we’re close, but I don’t know that we’ve had a super agent yet. Do you think that’s come, or are we close?

Jamie: If it has, it is so well designed we don’t see it — and I only mean that partly as a joke. I think there are frontier models that are clear market leaders and a variety of really capable systems. But I think it remains to be seen. The hype cycle is a lot more often tied to the funding cycle than to the technology development cycle. If there is something out there, we haven’t clearly identified it, and it’s operating at a level of transparency that is in a way both exciting and terrifying.

And I think they all create new problems. It’s iterative. If there were one uber model that really made the difference, it would create a whole bunch of other challenges and maintenance surfaces we haven’t identified yet. New failure modes and new design challenges we’d have to address.

Frank: Right. But luckily you help contribute to some of the solutions. I know you follow a system you call “write once, use everywhere” — a single project brief that feeds meeting notes, status reports, proposals, AI context, and so on. For someone listening who’s drowning in context switching, where does this start? How would you recommend they get going with this kind of scaffolding around AI?

Jamie: First, it’s about understanding what AI — or large language models specifically — are actually good at. They’re good at pattern recognition. They’re good at reproducing things. They’re good at iterating on things. So I try to identify a very small set of constant, nagging problems and build frameworks around them. Things like the project brief and meeting notes.

In the course of a day I might meet with six clients and need to be on top of each of those discussions. For years I’ve worked to keep consistent notes — a pretty structured notetaking process. AI allowed me to amplify that and make it a lot more efficient and consistent. The recommendation I give most people is: find one area like that, build a template around it, build some instructions around it, and iterate. Identify when it fails and how it fails, and then iterate to make it not fail that way again. Start working that one set of processes in a very focused way.

For me it’s the meeting cadence — identifying what my meetings are for the day, building templates for note capture, capturing notes, transcribing, and then a post-meeting workflow for cleaning up those notes, reading and summarizing them, and writing my own action items.

There’s another big failure mode here: using AI isn’t a problem when it gives you the wrong answer. It’s a big problem when it gives you the right answer and you stop looking. A lot of people are talking about this lately. The trap is to create a template and have the notes look really pretty — but never actually do the work of reading them or preparing for the next meeting. You end up with drift, where your own intelligence has less to access because you’ve offloaded a lot of cognitive processes to the machine, which is great at taking notes. But if you don’t read them, groom them, and go through them, you don’t know if they’re any good or useful for you as a person. That’s ultimately the context that matters.

As an engineer, on the other hand, my code has never been as well documented. I can give a coding assistant a very strict template for how I want my code documented, tell it to diagram or describe in very specific ways. It can hold a much larger context window than I can — simultaneously holding hundreds or thousands of lines of code and architecture in-memory in a way I can’t. That’s another place where it’s really useful. Find these well-structured, high-value places and let the assistant assist.

Frank: Makes a lot of sense. So a big part of your work has been with healthcare and nonprofits — organizations that are almost always under-resourced but carry enormous data responsibilities. In your experience, what are the specific failure modes you’ve most often seen as these organizations try to build or buy data systems?

Jamie: The primary failure mode I see in under-resourced organizations is largely related to operating from a scarcity mindset — which is challenging in the age of large language models. A lot of smaller nonprofit or underfunded organizations are really good at solving the same problem over and over again with whatever new tools they have on hand. That’s one of the reasons Excel is so consistent across these organizations — it’s accessible in a uniform way. And there are still organizations out there running key processes on someone’s undergrad Excel subscription.

Because they’re used to operating from scarcity, they burn a lot of resources doing the same thing over and over. In emergency management we used to talk about the life cycle of emergency management — preparation, response, recovery, and mitigation — going in a cycle. There’s a lot of money for response and recovery, but not always for mitigation and preparation because they’re not as visible. A lot of these organizations get stuck in just those two parts and never get to reintegrate the knowledge they gained from responding and recovering into formal systems or reproducible protocols. Some of that is practical — they don’t know if their software licenses will be there next year, so why design around them? But there’s a lot of energy expended in that cycle.

They also become dependent on vendor relationships with a lot of lock-in. Data sovereignty — being able to get the data out in a meaningful way — becomes a real issue. It all has a lot to do with funding, like everything else.

The other failure mode is thinking that non-technology skills aren’t useful to technology work. The most important skill I’ve learned as a consultant is something I learned from social workers: motivational interviewing. How do you get somebody to identify what the problem is, what the needed solutions are, and then come up with a reasonable plan for solving those things? That’s a really powerful thing. A lot of these organizations are run by social workers, run by people who have these skills. And they sometimes get so stuck in the obviously high-value areas — making sick people better, helping people, which is what they’re there for — that they assume the technology part is somehow special or outside their purview. It isn’t.

Frank: Related — you and I serve together on the board of CCNY, which among other services provides data capacity building for health and human service organizations in and around Buffalo. From your perspective, what does this look like on the ground? The need, the practice, the workflows?

Jamie: There are a lot of things I love about CCNY. If I have to narrow it down — one is that their staff sits at the intersection I was just talking about. They are excellent program builders, analysts, and statisticians, alongside people with significant frontline service delivery experience. So they understand both sides.

The other thing is that because they’re able to focus on program design, measurement, evaluation, and reporting, they have a little more time and consistency than organizations trying to straddle service delivery and reporting simultaneously. One of the nice things about moving from the ambulance to Medicaid work was just having more time to think about problems and look at things at a different scale. CCNY has the ability to work with multiple organizations, collect best practices, and encode that at a higher level — not stuck in immediate response mode or solving the immediate problem. They can look at things more broadly.

It allows them to train in different areas, focus on building better systems that not only help their team but help provide better service. It’s one of those rare organizations that bridges multiple disciplines productively without reducing the work to either one. It doesn’t try to solve every human services problem as if it’s a technology problem, or vice versa. There’s a really good community of practice there that drives a lot of knowledge exchange.

Frank: Yeah, that systems lens really positions them well as they begin to add more AI and automation. They’re a high-functioning team in my experience.

Okay, in closing — you’ve started a new blog, a new Substack, and in it you’ve written about approaching data systems by asking what it’s like to be the objects inside them. A SQL view, a spreadsheet, a color-coded cell. That’s not a typical consulting framework in my experience. Introduce the blog for us. Where does that come from, and how did it change what you see when you walk into a client engagement?

Jamie: Thanks, Frank. My Substack is called Copia — and it’s jamiebono.substack.com. Copia comes from Quintilian. It’s a rhetorical term meaning an abundance of preparation. It reflects my approach to context engineering and these other systems — we prepare a lot so that in the moment we can improvise. We can draw on the needed context from an abundant set of preparation.

The idea comes from object-oriented ontology — from philosophy — and this question of what it’s like to be a thing. I arrived at this approach because the problems people asked me to help solve were usually problems with things: a number on a spreadsheet, a report, a specific piece of code. It was helpful to look at the life cycle of that object — how it interacts with other systems, how it constrains or enables things.

One of the common examples is color-coded cells in Excel. For a person, color-coded labeling is really useful. But for the system, it’s challenging because it’s not easy to put into a database. Looking at where objects come into contact with other systems is really helpful for unpacking problems and finding solutions. A CFO sees one number, the operations team sees another, analytics is maintaining that number — but if you trace the root cause, it’s one query or one thing that moves through its own independent life cycle. It touches multiple departments, multiple concerns. Those things are usually not part of the equation. They’re usually seen as tools and not as actual constituents in the system.

Harman talks about how those objects withdraw — they’re not always easy to see. Part of my job as a consultant is to go in, talk to people, look at their systems, and draw those objects out. How does this object work? Where are the objects we’re not seeing? Where can we find opportunities to give those objects a better life cycle? And how does that then help the overall system?

My transition from EMS to Medicaid very clearly showed me that my perspective was partial, and that it was really useful to be able to dig in and see these other perspectives. The philosophy gives me vocabulary to talk about that. But at its core, it’s consulting — unpacking a complicated system and trying to track the life cycle of information and things through it.

Frank: Fascinating. And I had a couple of other thoughts when I read that. One: Samuel Beckett — also an English major like you — wrote Waiting for Godot, but he also wrote The Lost Ones, which is entirely about objects in a spherical space. That’s the whole story. It reminded me of your distillation of objects through a system.

The other thought: some of these objects will in effect become employees as agents take on full-scale roles. People will begin to think of them in your terms more because they’ll be doing jobs, doing task work that humans will be overseeing and managing. It’s very interesting to begin looking at it — almost personifying these objects and processes — because they’re going to be super important.

Jamie: And I don’t think it’s new. I don’t want to claim this is novel. It’s a problem we’ve always had in technical writing and technical communication — and it really becomes visible in disasters and emergencies. When systems fail, we often blame the documentation or the plan because it’s an object we can put blame on without moral judgment, as opposed to blaming the people who made choices in the moment.

Frank: Or as opposed to it being a black swan event where there’s no blame, because the whole event is a new experience.

Jamie: Right — with its own lifecycle, a beginning, middle, and end, and maybe not necessarily its own agency, but its own effect. And it’s interesting too when you start asking questions about how large language models and AI change things in the workforce. Ultimately, they are tools. We’ve talked a lot about them having agency and we’ve called them agents, which I think is a somewhat inconvenient nomenclature. But they are tools, designed to be used. When they fail, they become very visible.

The metaphor is apt, and it’s a useful way to look at these things. As a consultant it also sometimes lets me put blame on a view or a system rather than on the people who have to maintain it — people who have real opinions and feelings about it. It allows you to take a step back.

Frank: Right. And it also gives us a metaphor to the matrix. But with that — it’s a good place to park it. Jamie, it’s been really great having you here. You mentioned your Substack. Where else can listeners find you?

Jamie: All the usual places — LinkedIn, GitHub, Substack. Those are the places to find me.

Frank: Jamie, thanks so much. Love having you on.

Jamie: Thanks a lot, Frank. Appreciate you.