Taste in Software Development

Hi everyone. I'm Steven, and thanks for having me. So yeah, I'm going to be talking about taste and software development, though I did prepare some slides as well, where you can see that the actual subject is the future of software engineering. And I'll explain how this relates to taste in just a sec. But first, a bit about myself. I'm Steven, a CTO and co-founder of Cognition. And yeah, I've been a software engineer for probably like 20 years now. So very excited to be here speaking to you all.

As I mentioned at Cognition, taste in software development actually is just having a good opinion about the future of software engineering. And that's because what we build is a product called Devon. Devon is an AI software engineer, and what we sell is AI transformation for engineering organizations. So taste in software development for us is just understanding where the future is headed for AI coding and articulating that both in our product and to our customers. So, I want to talk a bit about how Cognition writes software as well, because I often get this question, which is, Cognition is a two-year-old company that has 50 engineers, and yet has half a billion of revenue and is valued at $26 billion. And it's a pretty small team for a company of this size. And I guess the answer is just Devon. Devon augments our engineering team so that even with a relatively lean engine team, we can do the work of much larger organization. And this comes in handy, because being the biggest users of our product, lets us kind of stay on top of it and make sure what we're selling to our customers is a great experience.

So I'm sure all of you have used AI coding tools before. So I want to talk a bit about the difference between Devon and other AI tools. So, to do that, I'm going to first recap a bit of the evolution of AI software engineering. We obviously started with tab completion. Shout out to GitHub Copilot, who invented that. And then we got chat with your IDE, like with products like cursor and Windsurf. And most recently, everyone's been using a lot more agents, such as cloud code, Codex, Devon, cursor, etc. So that's kind of where we are right now, we can delegate tasks to agents, and they will basically just complete them. AI has come a long way. But for users of Devon, they actually have a bit more than just that. They have what we call an AI software engineer, that they can assign not just tasks, but responsibilities to. So what does that look like? Well, first, at Cognition, like many of you, we use AI to write almost our code. And you can see that our merge rate has spiked pretty dramatically in the past couple months.

But I think what's more interesting is how we use Devon apart from implementation. So this is what I was alluding to when I say we don't just delegate tasks, we delegate responsibilities, much like how you might delegate a responsibility to a human software engineer. What that means is Devon is not just writing code, it's also reviewing code, testing and QAing features, and deploying and monitoring the rollout of these features. If there's ever a user bug or a production error, Devon will be the first responder to any incident, monitoring error streams and sentry errors. Setting up Devon to test and monitor all these aspects requires significant effort and is where we assist customers in achieving automation levels they desire.

A significant part of the work involves closing the loop, providing Devon with the necessary context for higher-level engineering duties. Users assign responsibilities to Devon through tools like the Datadog dashboard, instructing it to maintain certain metrics and respond to alerts. Devon autonomously reacts to alerts, digging into logs and recent commits to resolve issues promptly. This proactive approach is what users of Devon are currently benefiting from, realizing the future vision of AI software engineers handling tasks traditionally done by humans.

The future of software engineering evolves further as we predict that English will become the source of truth. Drawing an analogy to React in web development, where underlying HTML is generated through React programming, a similar shift towards using English as the primary language for coding might occur. This bold prediction extends beyond the current capabilities of AI software engineers and envisions a future where human-readable language becomes the cornerstone of software development, revolutionizing the coding process and accessibility for a broader audience.

And a lot of that work is what we call closing the loop. So just giving Devon the context it needs to do these higher level engineering duties. You can see in the slide, some screenshots we have in the middle of the Datadog dashboard, this is actually one of the main ways that you assign a responsibility to Devon is you just tell it, hey, these numbers, make sure they stay green. If they ever become red, then go fix it. And it does that by looking through logs, looking through recent commits, etc.

You can see in the right screenshot, an example of Devon automatically invoking itself based on an alert firing. So we have some Datadog alert based on LLM observability and Devon just automatically responds. In this case, it realized that this is an alert that actually fired already downstream of some other issue that already triaged. So yeah, this is what the users of Devon are actively experiencing today already. And for the longest time, this was the future of software engineering that we predicted.

Two years ago, we kind of went a bit viral when we first announced Devon as a concept and posted a demo on YouTube. And the future we were selling is just like, hey, one day you'll have AI software engineers, and they'll just do all the work that a human can do. And I would like to say that it's actually already come true. So for the rest of this talk, I actually wanted to make a second, maybe more bold prediction about the actual future of software engineering. And that is that English will become the source of truth.

So since this is a JS conference, I figured I'd make a React analogy, where with React, you can build websites. And the websites are still just HTML under the hood. But as a programmer, you get to write React. And the way it works is there is a reconciliation engine that takes the virtual DOM that's being returned by your React route render function, and then it figures out how to keep the actual DOM in sync with the virtual DOM. And this is an analogy for spec-driven development, where you might have a spec of your product. And every time you make a change to your spec, then a reconciliation engine, which will be an agent, will take that change in your spec and translate it to a change in your code.

This is the future we envision, where just as how React kind of let us move from working at the raw DOM level to a higher order programming language, English will be how we build apps in the future and build software in the future. And yeah, there's a lot of patterns where developers build frameworks so that they can write more expressive, higher-order languages that mirror their internal mental models better than the kind of routine syntax-level concerns.

There's a few problems we have to solve to actually get to this world where English is the source of truth. Just to name a few, it has to be stable, it has to be history-aware. It's kind of like you don't want, as soon as you change a spec, to regenerate the whole codebase from scratch. That would be very annoying. And you actually don't want a spec. I know that's crazy, because I just said everything's about a spec, but you want an implicit spec.

You want to be able to just talk to your agent, maybe have multiple conversations over the course of months, and all of that be combined as a raw primary source material that the spec is derived from. You don't actually want to interact with the literal spec. So this is another problem that has to be solved to actually get to this world. So yeah, just to conclude, two years ago, we claimed that the future of software engineering is that AI would write all your code. And that future has actually arrived. So now we're claiming a new future, which is English will be the source of truth. We believe this, and we're working really hard to build everything that's needed to get there.