Data Science, Python, SQL, Linux
Google's release of the QAT (Quantization-Aware Training) variants of Gemma 4 has changed the game for local inference on consumer hardware. By combining QAT with MTP (Multi-Token Prediction) , we can now run the heavy Gemma 4 26B MoE model at over 100 tokens/second on a single 12GB GPU. Here is a performance breakdown comparing various configurations on a consumer RTX 4070 setup. Performance…
Note: This post was drafted with significant AI assistance, synthesizing notes, bench results, and scripts from the l3ms homelab toolkit and the series of model-running posts on this site. The experiments, numbers, and failure modes documented here are real - the synthesis and prose are AI-assisted. Preface # Over the past year I've written posts on running gpt-oss-120b , Qwen3-Coder-Next , Gemma…
Update (May 20, 2026) : Gemma 4 MTP support has arrived in mainline llama.cpp via PR #23398 ! We are no longer dependent on the atomic-llama-cpp-turboquant fork. Update (May 21, 2026) : Successfully benchmarked Gemma 4 26B-A4B MTP on mainline llama.cpp using PR #23398 . On a 12GB RTX 4070, the mainline PR achieves ~52.4 tok/s (with f16 KV cache), which is a ~17% speedup over the non-MTP baseline…
I spent the last couple of days doing the extremely normal thing where you turn a trailer into 1,290 individual frames, send the frames through image models, stitch them back together at 12fps, and then slowly realize the actual boss fight is black bars. This is Loganime test v1 . It is not a polished short film. It is a debugging run that accidentally became cool. The repo behind it is NanoGhibli…
I had $300 API credits to burn and nothing else to show for it. That is only slightly unfair. I also got a folder full of tiny watercolor kitchens, cursed market maps, inconsistent character sheets, and a better sense of which image model I should bother when I need artifacts for this site. I have been experimenting with image generation models for blog images, visual headers, and the odd little…
Update (June 16, 2026) : Ran a comprehensive benchmark sweep on the latest mainline build of llama.cpp using a custom local benchmark harness comparing Q4_K_XL vs. Q6_K quant variants, and Thinking vs. NoThink modes. Key findings: Q4_K_XL MTP (n-max=2) is the speed champ at 60.3 tok/s (vs 52.1 tok/s baseline). Q6_K MTP (n-max=2) achieves 43.1 tok/s (vs 40.3 tok/s baseline). Disabling thinking mode…
I like Obsidian a lot. Probably too much. That is part of the problem. There is a certain kind of person, and I am very much one of them, who sees a flexible tool and immediately starts trying to make it the place where everything goes. Notes, obviously. Then projects. Then tasks. Then PDFs. Then health records. Then account lists. Then receipts. Then manuals. Then some weird idea that the whole…
Qwen3.6 continues the strong trend of highly capable open models, and the 35B-A3B MoE variant hits a sweet spot for consumer hardware. With a low active parameter count per token, it runs fast while still retaining the "smart" feel of larger reasoning models, and includes multimodal capabilities out of the box. This post covers my setup running Qwen3.6-35B-A3B on 12GB VRAM using mainline…
I thought this would take one evening. Open repo on the left. Agent on the right. Git somewhere visible. My own API key. Done. Instead I spent a week bouncing between editors, terminal apps, desktop wrappers, that slowly turned into me assembling an IDE out of separate tools like it was 2009 again. Update: 2026-08-09 # I have a tier list now. The custom Ghostty stack is still the best thing here,…
The hype around openclaw - and every other claw derivative - seems to be fading a bit as reality checks in. And what's interesting is the hype was never really about the product itself. It was about: The premise of personal agents working for you while you do other stuff The novelty of loosely defined specs and automations, as opposed to something structured like n8n Brilliant astroturfing by the…
Google DeepMind's Gemma 4 family is out-truly open with Apache 2 licenses, multimodal with support for text, image, and audio, and efficient enough to run on-device. The 26B-A4B variant is a mixture-of-experts model with only 4B parameters active per token, making it an ideal candidate for consumer hardware. This post covers my setup running Gemma 4 26B-A4B on 12GB VRAM using mainline llama.cpp,…
I've been a fan of WindowSwap for a while - the idea is simple and kind of beautiful. Someone points a camera out their window, streams it to the internet, and strangers get to borrow that view for a few minutes. A slice of someone else's world. The inspiration. Someone's window, somewhere in the world. (Pre-recorded, not live - mine actually is.) So I built my own. The Hardware # A Raspberry Pi…
When I moved away from Jekyll a couple years ago, I didn't expect to end up with something I'd name. It wasn't planned - it just evolved as I added features and tried to keep things simple. I now call it the NEAT framework: N etlify, E leventy, A lpine.js, and T ailwind CSS. The name fits twice over. It's an acronym, and it describes the philosophy: no heavy frameworks, no build drama, no…
I've been thinking about what kind of pages belong on a personal website that aren't blog posts and aren't portfolio entries. Something in between - deeper than a bookmark, more permanent than a tweet, more personal than a wiki article. The answer, for me, is a folio . What is a Folio? # Not a portfolio. Not a blog post. The word comes from bookbinding - a single sheet folded to make four pages, a…
A few weeks ago I came across this post on Marginalia showing that new Hacker News accounts are ten times more likely to use em dashes than older ones. Not as proof of anything - just as a signal. A stylistic fingerprint that's shifted measurably since large language models became mainstream. That felt like a project. What aidar does # aidar scans URLs and measures stylistic patterns that have…
I started at 3 PM and called it morning. I've been burning through AI tools faster than my brain can register. Exhausted Claude Code on my Pro subscription, exhausted Codex trial - done. Got access to Copilot CLI - exhausted the 300 premium requests. But the best part was getting access to AWS Bedrock hosted Claude Sonnet 4.6 - no more exhaustion. I finally understand why the creator of Claude…
You probably don't need >$100 plans for coding or toying with OpenClaw. My thoughts below. Note: Due to the mindbreaking speed this agent universe is moving at, this may be out of date within 2 days. (Last verified on 2026-04-19) What Changed Since Feb 2026 # Qwen Code: OAuth free tier was discontinued (2026-04-15). Use Alibaba Cloud Coding Plan, OpenRouter, Fireworks, or BYO API key instead.…
TL;DR # Hardware : i5-12600K (6P + 4E), RTX 4070 (12 GB), 64 GB DDR5 6000 MT/s, Linux (CachyOS, CUDA 13.0). Scripts : All bench and server scripts are in carteakey/l3ms under bench-models/ . Model : Qwen3-Coder-Next MXFP4 (~45 GB) - 80B total params, ~3B active per token. Result : ~40 t/s generation, 510+ t/s prompt processing. For comparison : GPT-OSS-120B on the same hardware gets ~25 t/s. Key…
Note: Writing unfiltered, raw and undeveloped thoughts. Lack of knowledge and experience may be more visible than usual. The year is 2023, GPT-4 has been launched. The world has suddenly gained interest in it's programming and generalization capabilities. You, as a techie, try it to generate some code. It performs the basic tasks just fine. You look 1 layer deep and ask more contextual questions,…
TL;DR: Perform exploratory data analysis on Databricks by "volleying" with AI agents like Claude or Copilot - iterate quickly with SQL queries, and generate notebooks automatically. The indeterministic nature of LLMs is great for looking at patterns within datasets and although it may lead to false smoking guns many times, there will be times where you'll find stuff that you didn't notice earlier.…
I had this old webcam lying around which I haphazardly bought during the "COVID webcam shortage" era. It was never great, but it worked fine for occasional video calls. Logitech C270 - aka the wall-eye, get it? Having nothing better to do on a weekend, I decided to repurpose it into a webcam server using a Raspberry Pi 4. Searching around, I found motion, a lightweight software that can turn a…
Gemini's Nano-banana is literally nuts. I'm surprised that it exists and we're so normalized to have such tools that are borderline magic. I recently got an idea of using it to restore old/damaged photos of my family. I called it "Personal Braumgartner Restoration" because of the channel Braumgartner Restoration on YouTube, who does amazing restorations of old paintings. Here is one of my photos…
Just out of curiosity, I wanted to see how well GPT‑OSS (the 120B variant) performs on the same coding‑related prompts that GPT‑5 highlights in its Demo . Scope and ground rules: Same prompt text wherever possible (copied from GPT‑5’s examples). Zero-shot - What you see is what you get, no follow-up prompting or corrections. Unquantized 120B running on my measly RTX 4070. See my previous post for…
Authorship note Parts of this post were drafted/refined with the help of gpt-oss-120b itself. How meta! TL;DR # Hardware : i5-12600K (6P + 4E), RTX 4070 (12 GB), 64 GB DDR5 RAM, Linux (CachyOS, CUDA 13.0). Result : 11 tokens/s -> 25 tokens/s generation, 420+ tokens/s prompt processing for 32k context. Biggest win : Enabling DDR5 XMP in BIOS. My RAM was running at 2000 MT/s instead of 6000 MT/s for…
Obsidian is one of the most useful pieces of software I've used. If someone asked me to choose only two apps for life, VSCode and Obsidian would be my picks. I've mostly been a multi-vault/multi-app note taker, treating each vault as a separate project or domain. The only issue with this approach for me was to think about where to put a note, and the constant context switching between vaults. I…
Note: This post was partially augmented by AI-specifically, Claude Sonnet 4. I used AI assistance to refine my initial draft, structure my ideas, and improve the writing. Given the subject matter, it felt appropriate (and meta 😃) to be transparent about this collaborative process. I spent two hours yesterday building six features for my website. I didn't write a single line of code. Instead, I…
With all respect to the incredible talent and creativity at Studio Ghibli and Miyazaki, I genuinely can’t stop myself from using ‘convert to ghibli’ - or as everyone now calls it, ‘ghiblify’ - on every image I see after the recent 4o image generation updates. At least, until it inevitably stops (perhaps rightly). It feels too good to merely call it inspiration. We are witnessing something…
I am a huge fan of self-hosting, and one of the things i use is Immich to store precious photos of me and my family. It gives me a bit of relief over paranoia of Google Photos / internet shutting down / hacked / outage stopping me from living in my nostalgia. A source of lot of these images are shared over messaging services e.g. WhatsApp. If you're an Indian (like me) - you'll be very aware of…
My blog uses ~~AI~~ , no seriously! Head to my homepage and you will find that the "quote of the day" is (or used to be) powered by ChatGPT. This site refreshes daily which ended up very silently leaking my whole 5$ of OpenAI API credit, leaving my blog stuck with a fallback, ultra-generic quote. Sad times! Instead of throwing more money into the OpenAI pit, I decided to spin up my own local LLM…
Warning: Borderline shitpost. Is there a thing as a favorite meme? Well i have one - its the "IQ Bell Curve Meme" - which unlike the name suggests, has nothing to do with IQ or intelligence. The X-axis represents any situation or concept that can progress from ignorance to knowledge back to ignorance, and the idea that simpler systems or explanations are often better, also known as the principle…
The thought of myself mindlessly scrolling through a world of increasingly diluted content, especially with the advent of generative AI, repulses me to no end. On the other hand, obsessively chasing productivity hacks, like pomodoro timers and blocked calendars, also has a somewhat similar effect. As always, I believe harmony lies in the middle, and that's what we should be striving to achieve.…
Meta recently released their CodeLlama models, which are fine tuned for code completion and analysis. The best thing about them is their crazy large context (upto 100,000 tokens), which would mean that one could input way larger codebases into the prompts and the models should be able to handle it. As such, they're a perfect drop-in replacement for Github Copilot (and even ChatGPT) for quick code…
So you have a Windows PC with WSL2 installed in it and want to SSH into it from another device like a Mac. The reason can be remote development, preference for a Linux environment, or any other. Sounds simple enough on paper, you SSH into your Windows machine and run wsl.exe - but that does not work. With the latest versions of WSL2, one is unable to access the wsl executable over ssh.…
I recently switched to a Macbook Air (M2) as my primary computer, leaving behind a loud and heavy ASUS-FX505 which I had bought as a gaming PC and served me well for over 3 years. It was still a quite capable device (Ryzen 3550h, 16gigs of RAM, SSD + 1 TB HDD), therefore it made sense to make it act as a secondary computing device, where I can offload my media and development computes. There is a…
This is in continuation of my older post. The site has come a long way from back when I posted the original article ( This is what it looked like). Here’s what I’ve been able to add so far. Tailwind CSS # I wanted to build my own minimal design system, so Tailwind seemed perfect for it. It abstracts CSS by providing much easier-to-understand utility classes. Yes, the class names become huge, but…
Fedora and openSUSE are removing H.264, H.265, and VC-1 VA-API video codecs support from Mesa to avoid potential patent issues. Why does it matter? # H26X's are currently the world’s most used HD video compression standards. Without Mesa supporting these codecs, any video playback will fallback to be CPU decoded, instead of GPU (which is inefficent, and may straight up not work on dated computers,…
So, after a lot of procrastination, I decided to migrate my old site - which was based on Jekyll's popular theme - Minimal Mistakes and hosted on GitHub Pages . While the experience of it had been great so far, it was time to move on. Here are a few reasons why: Setting up a local development environment in Jekyll is painful, to say the least. (Although docker does help in that) I am not a Rubyist…
Heroku is ending its free tier starting 28th November 2022 . It has always been the go-to free hosting solution for hobby devs and students for small & non-commercial projects. (RIP to a lot of tutorials that just went obsolete 😄) One of the best advantages of Heroku's free tier was the Postgres database it offered. What are the options? # For devs looking to migrate their apps & databases from…
Obsidian is a great note-taking application. It acts as your second brain, where notes are connected through links, much like how the human brain works. Moreover, it uses a local folder to store the notes in plain-text Markdown format, making it future-proof and giving total control over your knowledge base. Since the files are stored locally, you can use your methods of syncing these notes over…
The power of search engines is astonishing. You go to your favorite search engine, enter a few terms, and it magically crawls the entire internet for the most relevant search result for each query. With Google Search, Google has achieved verbal dominion over a whole category of digital experience, which is a rare feat for a tech business. A trademarked title becoming so ubiquitous that it has…