About coding and whatnot
Just couple of days ago poolside released their new model: Laguna S 2.1. I was happy to try it since it have native support for 1M tokens in context. I am doing development of hobby projects on my phone, via SSH prompting models to do stuff and 250k tokens of context using Qwen 3.6 27B Continue reading "Quick review of Laguna S 2.1"
I bought new phone and I started installing applications from previous device. When I installed termux I tried to copy ssh keys and ssh config from previous installation. This is where problems started. I could not reach any local address. Nothing in 10.x.x.x was working. No pings no ssh. I could do that to public Continue reading "Problem with local network in termux"
I remember when I started selfhosting my own instance of GitLab, which had simple code editor that allows you to edit code quickly inline. Then they integrated with fork of Visual Studio Code and allowed you to code in real repository directly in GL. I was thinking of doing quick code changes to applications at Continue reading "Remote phone development."
I do not know I find it funny. After few days fighting with local LLM to force it to write documentation and comments in code, in good quality I must add, and the forcing it to write readable unit tests, after I wrote: File is untracked (new). Build passes. Ready for review. I just answered. Continue reading "High Five with LLM"
Just out of curiosity I tested Qwen3.6-27B model with MTP speculative decoding with 6 draft tokens. Whole script looked like this: Those are results with vLLM bench tool: Not much better, despite GGUF MTP model having the same settings and being 6 times faster.
After optimizing lately my vllm startup I started optimizing performance of vLLM during inference. Thing it had abysmal performance spanning like 2-4ts. Or even 0.5ts with 180k context. That was really terrible and I started to think that I wont be able to get better performance out of Strix Halo inside my Desktop Framework. But I Continue reading "Lllama.cpp vs vLLM running Qwen 3.6 27B"
AI hype is calming down in some companies, but in others like in the one I am currently consulting for, we still on the hype. Few days ago I was asked by my peer about some problem with the tool that I was mostly developing. This is fine are there are always some bugs in Continue reading "Lets ask Claude!"
I have been running few coding agents in my spare time on my personal projects and Junie does not look great when paired with local model First of all, local models are capable but their ability to answer prompts in sustainable speeds degrades very quickly. I was experimenting with Qwen3.6-35B-A3B and Qwen3.5-35B-A3B. Both seems to Continue reading "Junie Coding Agent short review"
vLLM has very slow time to first token (TTFT). On AMD Ryzen AI Max+ 395 it can take even 10 minutes depending on exact settings and model size. To make my user experience of using those things myself I need to have comparison of llama.cpp and vLLM startup time and performance. But to do this Continue reading "Optimizing vLLM startup time"
I was playing over the weekend trying to write POC of C# interactive console. It is not ready yet but I wanted to have some cute hamster as logo for this project I will probably incorporate it to some degree into Hamster Wheel, so cute hamster seemed appropriate. Just for fun I decided to Continue reading "My new hobby: Asking LLMS to generate ASCII Hamsters"