AI testing professional and test automation expert.
If you’ve been following news on technical developments in AI, you’d have probably seen the term ‘evals’ suddenly showing up everywhere. In this post, we’ll unpack AI evaluations by comparing them with what we already know about software testing and proposing some ideas on how LLM based systems can be evaluated.
I have always been experimenting with how LLMs can help me learn fundamental programming concepts. As agents become more powerful, the fundamentals only increase in relevance. However, a recent study has shown how using coding agents impairs learning. The study goes on to show that the manner in which agents are used can either contribute to or impair learning. I see agents as both the problem and…
One of the most promising use cases for LLMs is to augment a model’s knowledge with custom data that can be used as the context for conversation. E.g., providing an LLM access to your company’s documentation and having your colleagues ask questions about this documentation.
The Greek hero Odysseus (the Romans called him Ulysses) was sailing back home from the Trojan war with his crew. En route, they had to sail through the Island of the Sirens.
This article is the written version of the talk Vincent Hennig and I presented at the WeAreDevelopers conference on 28 July, 2023 in Berlin.