Openclaw vs Hermes, 307 messages, 8 days, and one AI that thought past dates were "next."
Meet Suppandi. My AI assistant. Named after the lovable, well-meaning but spectacularly incompetent servant from the classic Indian comic strip.
The name turned out to be more prophetic than I intended.
If you have already met him in my previous articles, this time I tried a different hardware for him to run on and a different chat interface.
Chat Interface: Discord
Hardware: My home PC in Bengaluru. Not a cloud service. Not GPT-4. A humble little Qwen/Qwen3.5-4B model running locally on my AMD Ryzen 5 with a 6GB RTX 3050 GPU.
Tiny by today’s standards. Perfectly capable of surprising you in all the wrong ways.
The interesting part isn’t the model. The interesting part is what happens when you swap the agent framework around it.
I ran two different ones back to back:
Same model. Same computer. Same user. Very different experience.
I had another AI (Hello, Claude!) analyze 307 messages from those 8 days and produce a full visual breakdown. You can see the whole thing here.
Now, let’s talk about what stood out.
DISCLAIMER: This is not a very serious experiment, as you can see. I am just documenting my experience. Don’t mistake this for a serious peer-reviewed study, please 😁
This is the part that surprised me the most, and it’s the headline insight from the whole experiment.
With a small model on consumer hardware, the agent framework matters more than I expected. The model doesn’t change. What changes is whether you get a useful assistant or a polite paperweight.
OpenClaw’s philosophy is: never crash. It compacted context 6 times in 2 days, adding 10 to 91 seconds of overhead per compaction. Its mean response time was 52 seconds. Not because the model was slow. Because OpenClaw spent enormous energy tidying up after itself.
Hermes crashed 10 times, threw 29 “context at 100%” warnings, and occasionally got into loops where it would repeat “Let me...” four times in a single message. But it used 67 tool calls across 7 categories: terminal commands, browser navigation, file operations, code execution, and more.
When Hermes failed, it failed loudly and told you exactly why. When it worked, it actually did things.
“Crashing visibly and recovering quickly” beats “never crashing but never really working either.”
OpenClaw’s “stability” is mostly an illusion. Silent failures are harder to debug than loud ones.
I’d be doing you a disservice if I just talked about latency numbers.
I asked Suppandi: “When are the next 3 Amavasyas?” It answered confidently.
One of the dates was January 18, 2026. I asked this on March 29th. That date was two months in the past.
Suppandi searched a calendar, found the correct dates, and apologized with 🙏. Professional.
I asked Suppandi for help setting up headless Ubuntu remote desktop.
Its very first suggested command was sudo deluser root.
As in, delete the root user. On a remote headless machine. The one command guaranteed to lock me out of my own computer forever.
When confronted, Openclaw (or Qwen here) silently escaped by saying “context size exceeded”. I dont blame either of them for this. I blame ME, just ME.
Suppandi printed its full AgentMail API key in chat 5 separate times.
The “sharp servant with security awareness” was essentially shouting credentials into the void.
I sent Suppandi a test message through its AgentMail inbox: “Another test. Tell Vivek to drink water, Suppandi!”. The cron job Hermes had set up found the email. Reported it.
But, he never told me to drink water. The one task it received via email. Filed and forgotten.
If you’re tinkering with local AI agents (and you should be, it’s cheap and educational), my data says Hermes wins for actual daily use. It’s nearly 3x faster in clean conditions, it uses tools aggressively, and when it breaks you know what to fix.
OpenClaw’s “stability” is mostly an illusion. Silent failures are harder to debug than loud ones.
The full data breakdown with all the latency charts, verbosity stats, and the complete Comedy Hall of Fame is linked here.
The analysis covers everything from response time by query complexity to a “User Frustration Index” based on how many times I typed “Are you serious?” at my own computer. Spoiler: three times. In two days. On OpenClaw alone.
Thanks for reading Vivek’s GAG (Grounded AI Gazette)!
Up next: Showdown - Gemma 4 vs. Qwen 3.5 on a RTX 3050 (6GB VRAM)
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.