Recent content on My gists about Observability
On day 3 of the hackathon, I compared Opus 4.8 and Sonnet 4.6 using the gcx tool and debug-with-grafana skill to diagnose a fault-injected failure in the OpenTelemetry demo. Both models pinpoint the root cause correctly, while Sonnet proves cheaper for straightforward issues. And an reflections after 3 days, on how skill files and the right context make AI-driven observability much more effective.
Day 2 of the hackathon. I used the Grafana gcx debug-with-grafana skill with Claude to find the root cause of failing and slow services from metrics, logs and traces, and monitoring Claude’s own token usage and cost with OpenTelemetry. And the team outing with Quad driving and paintball.
Day 1 of the OpenValue hackathon in Belgium. I explored AI tooling and got hands-on with gcx, Grafana’s Observability as Code CLI tool.
Incident reports are often static and lack the detail needed for deep analysis. What if you would use Observable Framework to create interactive, data-driven incident reports. By using customer impact data from DownDetector, you can build reproducible and interactive incident reports that clearly share your line of thinking.
An Application Observability Code Challenge with virtual threads . Spot the difference between system thread and virtual thread usage for the same application with some problems.
The third and final day of the hackathon. During this hackathon I learned a lot about Quarkus, LangChain4j, Grafana MCP and many other things. Read this blog to read the progress of the last day and my conclusions of the hackathon.
The second day of the hackathon was a mix of activities. Next to the hackathon, we had a climbing outdoor activity. In this blog post I will share the results of the hackathon on day 2 and some pictures of the climbing activity.
Together with my colleagues of OpenValue Utrecht we are having a hackathon at a nice place in the south of Belgium. The goal of the hackathon is to build a monitoring buddy, a chatbot that can help you with your monitoring and observability questions. I will use Quarkus, LangChain4j, Tools and MCP in combination with Observability to build this chatbot. This blog post describes the progress on Day…
The first of the Application Observability Code Challenges step by step revealed. The application has some unexpected behaviour. In this post I will explain step by step what you can learn from the metrics and how you can improve the observability of the application to finally find the real problem.
The first of the Application Observability Code Challenges is now available. A simple Spring Boot application with a REST endpoint implemented in Jersey/JAX-RS This application has some unexpected behaviour. Can you find out what is happening based on the observability data? Can you improve the observability to prove that a fix really fixed the problems?
Application Observability Code Challenges are a way to challenge yourself to improve the observability of a sample application. Of course, this application has some surprises that you probably did not expect. These challenges will help you develop an observability mindset and become more familiar with your observability tools.
Your local development of a Grafana plugin or dashboard may lack representative data. Connecting to production data sources may be impossible, but there are other ways to connect. Read more to find out how to use Grafana as a proxy to access the data.
The power of observability becomes visible when you combine signals and put it all together. Checks performed by Checkly can provide a piece of the puzzle that gives you a better view of what is happening. It would be really nice if you could combine the data from Checkly in the same Grafana dashboards as your other observability data. This post describes how to bring it together and how to add…
Updates from Checkly Based on my previous post I have been in contact with Checkly about two parts that were not working as expected. I have some updates on these topics today. Sending traces from Checkly to Tempo In the previous post I mentioned that the traces in my Grafana environment showed root span not yet received. The root span was not available. This requires Checkly to send these spans…
Checkly This week I came across Checkly, a code-first synthetic monitoring solution that can do API checks and browser checks using PlayWright. It would have saved me a lot of time if this had been around 8 years ago. At that time I was working on a project where we were doing some external checks with a vendor like Checkly, but it was all manual configuration in a web interface. And internally we…
An announcement was made at GrafanaCON. Alloy is introduced in the family of Grafana tools. Alloy is an open source distribution of the OpenTelemetry Collector, but is will also replace Promtail. In the Observability Toolkit I use both Promtail and OpenTelemetry Collector, so it makes sense to merge them. In this blog post I will replace Promtail with Alloy. In another post I will see how…
Grafana Scenes can enhance your experience with Grafana dashboards by bringing observability data together and guiding users to the right data. Combining metrics, traces and logs helps to understand the actual behavior of a system. That is the goal of observability. If a dashboard or an app created with scenes does not give the right insights, then still the explore feature of Grafana is available…
Deep understanding of the actual behavior of a system is key for me. I like to know all the details because it helps me to see the whole picture. I am visually oriented, so tools like Grafana really help me in these situations. Observability is a topic that has come up in the last few years, but I have actually been doing this for years. But the good thing is that it is more standardized now. I…
One of the great things about OpenTelemetry is the standardisation of span attributes and resource attributes. An example of this is deployment.environment .
With Grafana Scenes it is possible to create more than just dashboards. There are options to create dashboards that guide the user. Deep dives with drill down pages can help to analyse problems. And with a feature like time range comparison it is even possible to use a feature that is not available with normal dashboards. But how do you start developing with Grafana Scenes? One trend is to develop…
As shown in my previous post you can add Grafana Faro to get more information about users who visit a website, in my case my own blog. This is how that setup looks like. But what kind of data is available now? Data sent by the browser The libraries from the Web SDK collect technical details about the browser interaction. The goal is not to track users, but to collect all sorts of technical data to…
If you use Hugo to create a blog or website, as I do, and you use GitHub Pages to host the blog, it’s hard to get observability signals in your usual observability stack. I have been using Grafana, Loki, Tempo and Prometheus for a long time, so using this stack makes sense to me. You can use Google Analytics with Hugo, but I don’t like third party cookies. If you search for Google…
I am passionate about Observability already for a long time. Monitoring tools have been helping me already for quite some years to get insights in the behavior of a service and especially a landscape of services. In 2016 I was working at a project where we were facing a number of problems and didn’t really have a clue what was going on in our application landscape. I suggested to work on…